-- Logs begin at Thu 2022-08-04 20:03:28 UTC, end at Thu 2022-08-04 22:12:38 UTC. -- Aug 04 21:20:27.628372 dv-cinder-852209-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 04 21:20:27.637131 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 04 21:20:27.639760 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 4 21:20:27 2022] *** Aug 04 21:20:27.639909 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 04 21:20:27.640024 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 04 21:20:27.640133 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: nodename: dv-cinder-852209-1-iscsi Aug 04 21:20:27.640244 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: machine: x86_64 Aug 04 21:20:27.640351 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: clock source: unix Aug 04 21:20:27.640452 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: pcre jit disabled Aug 04 21:20:27.640559 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: detected number of CPU cores: 4 Aug 04 21:20:27.640676 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: current working directory: / Aug 04 21:20:27.640676 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: detected binary path: /usr/bin/uwsgi-core Aug 04 21:20:27.640676 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: your processes number limit is 47680 Aug 04 21:20:27.640676 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: your memory page size is 4096 bytes Aug 04 21:20:27.640676 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: detected max file descriptor number: 2048 Aug 04 21:20:27.640676 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: lock engine: pthread robust mutexes Aug 04 21:20:27.641045 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: thunder lock: enabled Aug 04 21:20:27.641045 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 04 21:20:27.641150 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 04 21:20:27.680433 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: Python main interpreter initialized at 0x561c2a0159c0 Aug 04 21:20:27.680433 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: python threads support enabled Aug 04 21:20:27.680433 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: your server socket listen backlog is limited to 100 connections Aug 04 21:20:27.680433 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: your mercy for graceful operations on workers is 90 seconds Aug 04 21:20:27.684886 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: mapped 403077 bytes (393 KB) for 2 cores Aug 04 21:20:27.684886 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: *** Operational MODE: preforking *** Aug 04 21:20:27.684886 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: *** uWSGI is running in multiple interpreter mode *** Aug 04 21:20:27.684886 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: spawned uWSGI master process (pid: 91900) Aug 04 21:20:27.684886 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: spawned uWSGI worker 1 (pid: 91912, cores: 1) Aug 04 21:20:27.684886 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: spawned uWSGI worker 2 (pid: 91913, cores: 1) Aug 04 21:20:27.684886 dv-cinder-852209-1-iscsi devstack@keystone.service[91900]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 04 21:20:27.681504 dv-cinder-852209-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 04 21:20:33.334372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 21:20:33.334372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: from cryptography.utils import int_from_bytes Aug 04 21:20:33.335222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 21:20:33.335222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: from cryptography.utils import int_from_bytes Aug 04 21:20:33.335633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 21:20:33.335633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: from cryptography.utils import int_from_bytes Aug 04 21:20:33.335741 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 21:20:33.335741 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: from cryptography.utils import int_from_bytes Aug 04 21:20:33.385122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 21:20:33.385303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 21:20:33.385481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] command line args: [] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 21:20:33.385657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 21:20:33.385752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 21:20:33.385853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 21:20:33.385939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ================================================================================ {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 21:20:33.386068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] command line args: [] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 21:20:33.386173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] admin_token = **** {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.386341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 21:20:33.386439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] config_dir = [] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.386516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ================================================================================ {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 21:20:33.386593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.386733 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] config_source = [] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.386817 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] admin_token = **** {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.386893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] debug = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.386975 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] config_dir = [] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.387091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.387351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.387351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] config_source = [] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.387466 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.387530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] debug = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.387665 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.387770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.387900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.388004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.388105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.388197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.388295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] list_limit = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.388397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.388490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] log_config_append = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.388580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.388683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.388782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] list_limit = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.388886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] log_dir = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.388991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] log_config_append = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.389084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] log_file = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.389175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.389284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.389407 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] log_dir = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.389488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.389608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] log_file = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.389688 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.389815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.389901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.389980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.390138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.390240 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.390312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.390438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.390521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.390636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.390709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.390824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.390898 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.391021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.391097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.391209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.391283 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.391400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.391636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.391728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.391878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.391957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.392078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.392203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.392291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.392505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.392697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.392774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.392900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] publish_errors = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.393095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.393263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.393339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.393499 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] publish_errors = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.393575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.393675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.393751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.393908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.394058 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.394189 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.394313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] standard_threads = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.394431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.394542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.394666 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.394778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.394778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.394995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] standard_threads = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.395738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] use_journal = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.395864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.396010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] use_json = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.396161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.396275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] use_stderr = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.396392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.396505 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] use_syslog = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.396616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] use_journal = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.396736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.396848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] use_json = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.396848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] use_stderr = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.397057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.397168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] use_syslog = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.397278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.397468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 21:20:33.397583 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.397694 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.397812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.397924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.398059 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.398183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.398296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.398412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.398524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.398635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.398751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.398871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.398990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.399108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.399221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.399331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.399445 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.399565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.399677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.399787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.399898 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.399898 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.400114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.400114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.400343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.400458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.400570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.400570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.400794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.400906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.401023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.401141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.401259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.401377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.401497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.401705 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.401830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.401943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.402088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.402209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.402330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.402443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.402553 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.402670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.402787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.402905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.403091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.403206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.403317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.403436 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.403547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.403664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.403787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.403900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.404018 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.404129 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.404239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.404350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.404472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.404590 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.404707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.404825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.404936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.405047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.405168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.405286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.405382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.405476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.405575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.405669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.405770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.405863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.405955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.406082 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.406181 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.406285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.406391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.406507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.406618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.406762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.406885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.407009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.407122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.407223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.407333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.407444 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.407556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.407676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.407788 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.407906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.408017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.408128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.408128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.408342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.408455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.408580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.408580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.408790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.408790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.409005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.409117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.409236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.409349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.409465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.409576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.409688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.409799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.409918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.410057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.410172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.410284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.410395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.410519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.410680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.410801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.410913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.411081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.411202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.411324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.411438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.411438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.411663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.411663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.411874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.411976 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.412095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.412208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.412320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.412439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.412552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.412664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.412795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.412910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.413011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.413127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.413239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.413351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.413470 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.413584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.413694 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.413813 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.413925 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.414088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.414280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.414397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.414515 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.414628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.414739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.414851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.414972 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.415086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.415198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.415310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.415422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.415542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.415663 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.415784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.415897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.416011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.416123 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.416246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.416360 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.416483 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.416597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.416710 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.416821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.416933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.417046 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.417166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.417280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.417404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.417517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.417629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.417741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.417860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.417974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.418110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.418224 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.418336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.418448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.418572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.418685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.418797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.418915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.419028 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.419141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.419261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.419382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.419500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.419619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.419732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.419842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.419962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.420075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.420187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.420300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.420418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.420537 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.420658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.420771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.420883 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.420994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.421112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] auth.external = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.421225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.421345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.421458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.421570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.421683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.421794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.421905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.422038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] auth.password = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.422186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] auth.external = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.422314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] auth.token = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.422428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.422540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.422540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.422751 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.422870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.423034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.423155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.423269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] auth.password = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.423381 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.423573 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] auth.token = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.423691 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.423812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.423927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.424040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.424159 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] credential.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.424273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.424386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.424507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.424621 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.424740 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.424855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.424967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.425086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.425208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.425322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.425436 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] credential.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.425548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.425660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.425785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.425899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.426055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.426171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.426324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.426445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.426571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.426694 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.426808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.426921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.427033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.427145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.427267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.427381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.427495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.427608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.427720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.427833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.428017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] federation.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.428133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.428246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.428358 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.428470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.428583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.428704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.428818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.428930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.429042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.429155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.429267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] federation.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.429388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.429507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.429620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.429733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.429932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.430065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.430195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.430312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.430425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.430577 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.430695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.430808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.430929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.431044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.431156 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.431275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.431389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.431501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.431622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.431738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.431850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.431969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.432084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.432196 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.432317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.432431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.432543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.432655 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.432655 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.432876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.432997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.433111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.433223 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.433335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.433455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.433568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.433689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.433804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.433917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.433917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.434166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.434281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.434401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.434515 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.434627 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.434739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.434852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.434964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.435085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.435206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.435319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.435431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.435543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.435669 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.435783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.435897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.436009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.436121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.436233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.436353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.436467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.436578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.436690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.436803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.436916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.437035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.437148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.437267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.437379 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.437490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.437601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.437721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.437834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.437946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.438072 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.438187 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.438309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.438431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.438550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.438663 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.438776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.438893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.439006 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.439126 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.439240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.439352 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.439468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.439580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.439691 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.439811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.439924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.440036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.440195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.440310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.440429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.440551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.440664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.440776 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.440888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.441000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.441113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.441237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.441356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.441469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.441581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.441692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.441804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.442527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.442676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.442822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.442937 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.443049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.443170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.443293 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.443408 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.443408 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.443620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.443732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.443844 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.443844 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.444066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.444066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.444279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.444392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.444504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.444626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.444740 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.444859 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.444972 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.445084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.445196 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.445318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.445432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.445544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.445544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.445754 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.445866 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.445987 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.446120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.446243 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.446359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.446471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.446583 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.446704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.446819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.446931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.446931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.447141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.447252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.447373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.447488 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.447600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.447712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.447824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.447942 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.448064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.448064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.448276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.448388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.448501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.448622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.448736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.448848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.448961 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.449073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.449185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.449306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.449419 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.449533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.449644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.449757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.449872 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.449994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.450163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.450276 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.450388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.450501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.450613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.450806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.450923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.451035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.451147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.451259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.451371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.451492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.451607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.451726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.451840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.451952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] resource.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.452064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.452185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.452299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.452412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.452523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.452635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.452747 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.452867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.452981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.453100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.453213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] resource.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.453326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.453439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.453559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.453674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.453794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.453906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.454035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.454151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.454275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] role.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.454390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.454502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] role.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.454615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.454727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.454839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.454960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.455074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.455186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.455299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.455411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.455529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] role.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.455651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.455765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] role.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.455878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.455990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.456102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.456214 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.456334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.456448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.456561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.456672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.456784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.456897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.457018 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.457132 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.457244 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.457356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.457468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.458174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.458361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.458481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.458594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.458707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.458818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.458941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.459055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.459167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.459279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.459391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.459504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.459628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.459742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.459854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.459970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.460085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.460085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.460304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.460418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.460530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.460641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.460753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.460865 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.460985 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.461100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.461212 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.461323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.461436 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.461547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.461667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.461781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.461893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.462033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.462150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.462263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.462391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.462505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.462607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.462717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.462830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.462941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.463066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] token.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.463181 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.463336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.463467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.463581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.463693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.463814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.463928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.464041 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.464154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] token.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.464267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.464392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.464518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.464633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.464745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.464857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.464969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.465082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.465203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.465317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.465430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.465543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.465656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.465768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.465889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.466022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.466138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.466251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.466364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.466477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.466602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.466716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.466829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.466942 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.467054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.467175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.467330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.connection = **** {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.467460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.467582 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.467697 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.467809 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.467921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.468033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.468154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.468268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.468414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.connection = **** {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.468540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.468664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.468778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.468891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.469012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.469127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.469239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.469352 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.469495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.469611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.469794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.469915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.470046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.470161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.470275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.470387 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.470550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.470672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.470794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.470908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471021 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91913) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 21:20:33.471924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 21:20:33.471924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91912) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 21:20:33.499527 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.499755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.499918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.500002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.500154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.500381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.508907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.509576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.539644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.539876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.540071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.540308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.593012 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.593301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.644022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.644256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.644578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.644816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.710763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.710994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.711106 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.711219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.711304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.711440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.711523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.711658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.711742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.711874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.711956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.712089 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.735212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.735422 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.735523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.735655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.735748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.735869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.735965 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.736104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.736196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.736298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.736382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.736510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.736593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.736721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.748219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.748331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.761527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.761722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.761812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.761944 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.762050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.762202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.762301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.762423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.762505 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.762646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.762730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.762844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.762929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91912) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.763053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91913) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 21:20:33.779565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.779777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 21:20:33.780029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.780133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.780239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 21:20:33.780322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 21:20:33.780561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.780643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.780759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 21:20:33.780842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.781024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.781105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.781279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.781363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.781529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.781778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.791002 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.791279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.791541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.791639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.791815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.791898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.792080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.792355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.798714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.799228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.804085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.804360 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.804531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.804639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.804809 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.804919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.805086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.805197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.805366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.805456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.805651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.805781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.805905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.806067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.806189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.806273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.806456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.806537 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.806719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.806800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.806984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.807242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.829783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.830554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.831380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.831660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.832035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.832323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.836896 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.837178 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.837460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.837573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.837851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.838155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.855538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.855538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.855789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.855789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.856033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.856033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.856033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.856033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.856413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.856531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.856612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.856685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.856766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.856841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.856912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.856990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.857063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.857063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.857063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.857063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.867196 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.867475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.868130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.868403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.871448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.872411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.874585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.874881 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.875537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.875827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.879277 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.880378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.882605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.883835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.885777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.886095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.886492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.886690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.890314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.890314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.890624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.890624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.890624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.890624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.890624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.890624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.903761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.904047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.904312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.904586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.905307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.905589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.905853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.906150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.911562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.913108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.913193 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.913383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.914681 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.915003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.916248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.916528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.916830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.917108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.917385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.917796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.918099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.918398 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.918677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.918962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.929356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.929630 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.929906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.930213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.930490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.930772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.930855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.931047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.931168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.931350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.931623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.931895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.932171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.932454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.946796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.948094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.949787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.951065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.952768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.953994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.955294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.956483 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.956818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.957118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.957400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.958500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.958887 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.959182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.964418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.965591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.966048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.967220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.968996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.969284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.969569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.969840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.970156 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.970443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.970722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.970993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.978144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.978429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.978705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.979086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.979371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.979660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.985824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91912) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.986126 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.986408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.986683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.986768 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91913) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 21:20:33.986957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.987040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.987227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.987310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.987499 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.987594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.987787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.987868 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.988046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.988175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.988310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.988450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.988586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.988723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.989056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.989193 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.989335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.989469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.989604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.989735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91912) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.989859 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.990211 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:33.990475 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91913) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 21:20:34.023539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x561c2a0159c0 pid: 91912 (default app) Aug 04 21:20:34.024793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x561c2a0159c0 pid: 91913 (default app) Aug 04 21:20:34.026871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878d72-9884-40a8-b91d-d456a1c63cc5 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:34.027167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878d72-9884-40a8-b91d-d456a1c63cc5 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:34.027463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878d72-9884-40a8-b91d-d456a1c63cc5 None None] PATH_INFO: `/v3` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:34.031753 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1/1] 10.222.0.73 () {50 vars in 791 bytes} [Thu Aug 4 21:20:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:42.686702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcecee1-d2df-41ca-a5e8-a347b5e8bf05 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:42.687230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcecee1-d2df-41ca-a5e8-a347b5e8bf05 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:42.687230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcecee1-d2df-41ca-a5e8-a347b5e8bf05 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:42.691651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1/2] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:20:42.697703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf53fa-9d0e-40b8-9e0f-7c1f5c469d34 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:42.698161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf53fa-9d0e-40b8-9e0f-7c1f5c469d34 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:42.698309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf53fa-9d0e-40b8-9e0f-7c1f5c469d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:42.770572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: INFO dbcounter [None req-daaf53fa-9d0e-40b8-9e0f-7c1f5c469d34 None None] Registered counter for database keystone Aug 04 21:20:42.789758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG oslo_db.sqlalchemy.engines [None req-daaf53fa-9d0e-40b8-9e0f-7c1f5c469d34 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91912) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 21:20:42.796045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writer thread running {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 21:20:42.983209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG passlib.handlers.bcrypt [None req-daaf53fa-9d0e-40b8-9e0f-7c1f5c469d34 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91912) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 21:20:42.983866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG passlib.handlers.bcrypt [None req-daaf53fa-9d0e-40b8-9e0f-7c1f5c469d34 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91912) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 21:20:43.042514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-daaf53fa-9d0e-40b8-9e0f-7c1f5c469d34 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:43.044069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-daaf53fa-9d0e-40b8-9e0f-7c1f5c469d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:43.183240 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 2/3] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 04 21:20:43.304224 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcecee1-d2df-41ca-a5e8-a347b5e8bf05 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:43.374778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: INFO dbcounter [None req-afcecee1-d2df-41ca-a5e8-a347b5e8bf05 None None] Registered counter for database keystone Aug 04 21:20:43.397428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG oslo_db.sqlalchemy.engines [None req-afcecee1-d2df-41ca-a5e8-a347b5e8bf05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91913) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 21:20:43.403855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writer thread running {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 21:20:43.527426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcecee1-d2df-41ca-a5e8-a347b5e8bf05 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:43.531767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9b6c0-dede-481b-990d-414e65e1e368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:43.532403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9b6c0-dede-481b-990d-414e65e1e368 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:43.532684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9b6c0-dede-481b-990d-414e65e1e368 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:43.532960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9b6c0-dede-481b-990d-414e65e1e368 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:43.542554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9b6c0-dede-481b-990d-414e65e1e368 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:43.594537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d7d9b6c0-dede-481b-990d-414e65e1e368 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 21:20:43.594797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9b6c0-dede-481b-990d-414e65e1e368 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:43.605731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 2/4] 10.222.0.73 () {58 vars in 1324 bytes} [Thu Aug 4 21:20:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:20:43.611962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf53fa-9d0e-40b8-9e0f-7c1f5c469d34 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:43.626415 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf53fa-9d0e-40b8-9e0f-7c1f5c469d34 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:43.630967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439069d-50a4-4550-8428-b3d348959e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:43.632107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439069d-50a4-4550-8428-b3d348959e2d admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:43.632392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439069d-50a4-4550-8428-b3d348959e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:43.632665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439069d-50a4-4550-8428-b3d348959e2d admin admin] PATH_INFO: `/v3/projects/1a7e2e1e30694bd4b3a5931f10dc9fa8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:43.635167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439069d-50a4-4550-8428-b3d348959e2d admin admin] RBAC: Authorizing `identity:get_project(project_id=1a7e2e1e30694bd4b3a5931f10dc9fa8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:43.686384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9439069d-50a4-4550-8428-b3d348959e2d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 21:20:43.686725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439069d-50a4-4550-8428-b3d348959e2d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:43.691554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 3/5] 10.222.0.73 () {56 vars in 1207 bytes} [Thu Aug 4 21:20:43 2022] GET /identity/v3/projects/1a7e2e1e30694bd4b3a5931f10dc9fa8 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:45.688736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b0cd6-f43a-452e-8508-6517b3d80cb9 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:45.689352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b0cd6-f43a-452e-8508-6517b3d80cb9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:45.689352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b0cd6-f43a-452e-8508-6517b3d80cb9 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:45.694154 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 3/6] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:20:45.700575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e16af-315f-4693-99fd-ce8823777879 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:45.701093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e16af-315f-4693-99fd-ce8823777879 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:45.701187 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e16af-315f-4693-99fd-ce8823777879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:45.721679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a98e16af-315f-4693-99fd-ce8823777879 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:45.723467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a98e16af-315f-4693-99fd-ce8823777879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:45.731616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 4/7] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 21:20:45.754102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b0cd6-f43a-452e-8508-6517b3d80cb9 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:45.767010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b0cd6-f43a-452e-8508-6517b3d80cb9 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:45.771247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df91ee5-8e09-409f-8de8-d2286f2b2ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:45.771847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df91ee5-8e09-409f-8de8-d2286f2b2ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:45.772176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df91ee5-8e09-409f-8de8-d2286f2b2ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:45.772402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df91ee5-8e09-409f-8de8-d2286f2b2ef1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:45.780091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df91ee5-8e09-409f-8de8-d2286f2b2ef1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:45.781865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df91ee5-8e09-409f-8de8-d2286f2b2ef1 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:45.791059 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 4/8] 10.222.0.73 () {58 vars in 1324 bytes} [Thu Aug 4 21:20:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:20:45.796635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e16af-315f-4693-99fd-ce8823777879 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:45.806267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e16af-315f-4693-99fd-ce8823777879 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:45.810711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3d5c4-5f18-4495-a79b-7e0e5b87f698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:45.811346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3d5c4-5f18-4495-a79b-7e0e5b87f698 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:45.811630 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3d5c4-5f18-4495-a79b-7e0e5b87f698 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:45.811909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3d5c4-5f18-4495-a79b-7e0e5b87f698 admin admin] PATH_INFO: `/v3/users/ec8458e10b6c4d69ba0ad246e923be31` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:45.813279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3d5c4-5f18-4495-a79b-7e0e5b87f698 admin admin] RBAC: Authorizing `identity:get_user(user_id=ec8458e10b6c4d69ba0ad246e923be31)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:45.815153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3d5c4-5f18-4495-a79b-7e0e5b87f698 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:45.816692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 5/9] 10.222.0.73 () {56 vars in 1198 bytes} [Thu Aug 4 21:20:45 2022] GET /identity/v3/users/ec8458e10b6c4d69ba0ad246e923be31 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:47.892229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477e94c-6f8f-4a2e-b860-178966718d5e None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:47.892725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477e94c-6f8f-4a2e-b860-178966718d5e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:47.892725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477e94c-6f8f-4a2e-b860-178966718d5e None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:47.894040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 5/10] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:20:47.899741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269b49e-937c-4248-beea-aad724415d14 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:47.902196 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269b49e-937c-4248-beea-aad724415d14 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:47.902196 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269b49e-937c-4248-beea-aad724415d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:47.920301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-7269b49e-937c-4248-beea-aad724415d14 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:47.921514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-7269b49e-937c-4248-beea-aad724415d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:47.978593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20314a99-c11b-4c80-b16d-a3586f35d1d5 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:47.978882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20314a99-c11b-4c80-b16d-a3586f35d1d5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:47.979157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20314a99-c11b-4c80-b16d-a3586f35d1d5 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:47.980463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 6/11] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:20:47.985663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a3de1-1a2d-4b3a-9e9f-62f013c57070 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:47.985947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a3de1-1a2d-4b3a-9e9f-62f013c57070 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:47.986261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a3de1-1a2d-4b3a-9e9f-62f013c57070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:48.003388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 6/12] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:20:48.026475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269b49e-937c-4248-beea-aad724415d14 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:48.028628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG passlib.handlers.bcrypt [None req-c10a3de1-1a2d-4b3a-9e9f-62f013c57070 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91913) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 21:20:48.029058 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG passlib.handlers.bcrypt [None req-c10a3de1-1a2d-4b3a-9e9f-62f013c57070 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91913) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 21:20:48.050488 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:48.074599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c10a3de1-1a2d-4b3a-9e9f-62f013c57070 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:48.079585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c10a3de1-1a2d-4b3a-9e9f-62f013c57070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:48.135167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269b49e-937c-4248-beea-aad724415d14 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:48.169460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3333410-ea34-449e-a254-ffa7eb0ea25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:48.170044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3333410-ea34-449e-a254-ffa7eb0ea25f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:48.170332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3333410-ea34-449e-a254-ffa7eb0ea25f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:48.170606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3333410-ea34-449e-a254-ffa7eb0ea25f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:48.176751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 7/13] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:20:48.181918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3333410-ea34-449e-a254-ffa7eb0ea25f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-f3333410-ea34-449e-a254-ffa7eb0ea25f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:20:48.190333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:20:48.191431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:20:48.192400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 7/14] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:20:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:20:48.197288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a3de1-1a2d-4b3a-9e9f-62f013c57070 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:48.198622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3333410-ea34-449e-a254-ffa7eb0ea25f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:48.278126 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:48.294770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3333410-ea34-449e-a254-ffa7eb0ea25f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:48.295744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a3de1-1a2d-4b3a-9e9f-62f013c57070 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:48.298489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:48.329185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa294721-e3f9-42ab-94f1-0efa539f3cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:48.329808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa294721-e3f9-42ab-94f1-0efa539f3cc9 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:48.330119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa294721-e3f9-42ab-94f1-0efa539f3cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:48.330394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa294721-e3f9-42ab-94f1-0efa539f3cc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:48.333435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eae2d-103b-4bac-b9b7-3aa39842e89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:48.334412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eae2d-103b-4bac-b9b7-3aa39842e89e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:48.334699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eae2d-103b-4bac-b9b7-3aa39842e89e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:48.334978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eae2d-103b-4bac-b9b7-3aa39842e89e None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:48.335874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eae2d-103b-4bac-b9b7-3aa39842e89e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:20:48.336181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eae2d-103b-4bac-b9b7-3aa39842e89e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:48.337614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eae2d-103b-4bac-b9b7-3aa39842e89e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:48.350733 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 8/15] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:20:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:48.356403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eae2d-103b-4bac-b9b7-3aa39842e89e None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:48.398440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa294721-e3f9-42ab-94f1-0efa539f3cc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:48.399923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa294721-e3f9-42ab-94f1-0efa539f3cc9 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:48.452666 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eae2d-103b-4bac-b9b7-3aa39842e89e None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:48.481294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:48.486924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d841b-314a-4b93-8b0c-df4550c87dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:48.487839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d841b-314a-4b93-8b0c-df4550c87dab None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:48.488072 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d841b-314a-4b93-8b0c-df4550c87dab None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:48.488347 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d841b-314a-4b93-8b0c-df4550c87dab None admin] PATH_INFO: `/v3/users/ec8458e10b6c4d69ba0ad246e923be31` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:48.490655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d841b-314a-4b93-8b0c-df4550c87dab None admin] RBAC: Authorizing `identity:get_user(user_id=ec8458e10b6c4d69ba0ad246e923be31)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:48.492092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d841b-314a-4b93-8b0c-df4550c87dab None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:48.499775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 9/16] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:20:48 2022] GET /identity/v3/users/ec8458e10b6c4d69ba0ad246e923be31 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:20:48.499775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d841b-314a-4b93-8b0c-df4550c87dab None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:48.500328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 8/17] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:20:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 21:20:48.505677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa294721-e3f9-42ab-94f1-0efa539f3cc9 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:48.537225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:48.607632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d841b-314a-4b93-8b0c-df4550c87dab None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:48.643486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2785d-da21-484b-8d00-a2572dc76c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:48.644147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2785d-da21-484b-8d00-a2572dc76c3c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:48.644432 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2785d-da21-484b-8d00-a2572dc76c3c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:48.644708 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2785d-da21-484b-8d00-a2572dc76c3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:48.646775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2785d-da21-484b-8d00-a2572dc76c3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:48.648251 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2785d-da21-484b-8d00-a2572dc76c3c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:48.649858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 10/18] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:20:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:20:48.655615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2785d-da21-484b-8d00-a2572dc76c3c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:48.718795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa294721-e3f9-42ab-94f1-0efa539f3cc9 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:48.753355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d067a-4bd0-4404-baba-183671b0244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:48.754049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d067a-4bd0-4404-baba-183671b0244e None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:48.754921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d067a-4bd0-4404-baba-183671b0244e None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:48.755201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d067a-4bd0-4404-baba-183671b0244e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:48.758651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2785d-da21-484b-8d00-a2572dc76c3c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:48.760145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:48.765381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d067a-4bd0-4404-baba-183671b0244e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:48.766952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d067a-4bd0-4404-baba-183671b0244e None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-8d4d067a-4bd0-4404-baba-183671b0244e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:20:48.783312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:20:48.784191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:20:48.784989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:20:48.784989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:20:48.784989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:20:48.784989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:20:48.784989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 9/19] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:20:48.790411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d067a-4bd0-4404-baba-183671b0244e None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:48.792968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b930458-b7a4-4d8a-840d-0336308ba354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:48.793594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b930458-b7a4-4d8a-840d-0336308ba354 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:48.793879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b930458-b7a4-4d8a-840d-0336308ba354 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:48.794184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b930458-b7a4-4d8a-840d-0336308ba354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:48.794975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b930458-b7a4-4d8a-840d-0336308ba354 None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.domain.id=default, user.id=ec8458e10b6c4d69ba0ad246e923be31) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:20:48.795282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b930458-b7a4-4d8a-840d-0336308ba354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:48.796723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b930458-b7a4-4d8a-840d-0336308ba354 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:48.805592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 11/20] 10.222.0.73 () {56 vars in 1322 bytes} [Thu Aug 4 21:20:48 2022] GET /identity/v3/role_assignments?user.id=ec8458e10b6c4d69ba0ad246e923be31&scope.domain.id=default&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:20:48.827096 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:48.899506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d067a-4bd0-4404-baba-183671b0244e None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:48.935422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f0a0e-ed33-44b2-acad-28271f8ac8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:48.936121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f0a0e-ed33-44b2-acad-28271f8ac8f0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:48.936324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f0a0e-ed33-44b2-acad-28271f8ac8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:48.936599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f0a0e-ed33-44b2-acad-28271f8ac8f0 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:48.937304 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f0a0e-ed33-44b2-acad-28271f8ac8f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:20:48.937611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f0a0e-ed33-44b2-acad-28271f8ac8f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:48.939072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f0a0e-ed33-44b2-acad-28271f8ac8f0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:48.956999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 10/21] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:20:51.117914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc7796-a8d8-4c51-8dcb-f69e560950b0 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:51.118984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc7796-a8d8-4c51-8dcb-f69e560950b0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:51.119479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc7796-a8d8-4c51-8dcb-f69e560950b0 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:51.121095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 12/22] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:20:51.126398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96a39f-3305-42e8-ab43-2cbf63efafb8 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:51.127225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96a39f-3305-42e8-ab43-2cbf63efafb8 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:51.127714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96a39f-3305-42e8-ab43-2cbf63efafb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:51.150988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-5b96a39f-3305-42e8-ab43-2cbf63efafb8 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:51.152552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5b96a39f-3305-42e8-ab43-2cbf63efafb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:51.240741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 11/23] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:20:51.274374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc7796-a8d8-4c51-8dcb-f69e560950b0 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:51.325899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139174-fa00-436a-9cc4-c179c52854b9 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:51.326529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139174-fa00-436a-9cc4-c179c52854b9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:51.326996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139174-fa00-436a-9cc4-c179c52854b9 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:51.328504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 12/24] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:20:51.333488 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd5a9-dd07-468b-bbf4-0d62e170b543 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:51.334040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd5a9-dd07-468b-bbf4-0d62e170b543 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:51.334543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd5a9-dd07-468b-bbf4-0d62e170b543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:51.367815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-5fdfd5a9-dd07-468b-bbf4-0d62e170b543 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:51.374545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5fdfd5a9-dd07-468b-bbf4-0d62e170b543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:51.458449 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:51.488982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc7796-a8d8-4c51-8dcb-f69e560950b0 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:51.557587 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 13/25] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:20:51.584651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3d561-34d1-41b4-adfc-bdad9ce1c359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:51.585319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3d561-34d1-41b4-adfc-bdad9ce1c359 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:51.585605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3d561-34d1-41b4-adfc-bdad9ce1c359 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:51.585882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3d561-34d1-41b4-adfc-bdad9ce1c359 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:51.597445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfd5a9-dd07-468b-bbf4-0d62e170b543 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:51.610927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3d561-34d1-41b4-adfc-bdad9ce1c359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-f4d3d561-34d1-41b4-adfc-bdad9ce1c359 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:20:51.620554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:20:51.621474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:20:51.622245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 13/26] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:20:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:20:51.628506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3d561-34d1-41b4-adfc-bdad9ce1c359 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:51.682357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:51.804514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfd5a9-dd07-468b-bbf4-0d62e170b543 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:51.809576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3d561-34d1-41b4-adfc-bdad9ce1c359 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:51.860884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be01a3-fde1-439b-bb7b-e55ce0d8aa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:51.861860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be01a3-fde1-439b-bb7b-e55ce0d8aa01 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:51.862358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be01a3-fde1-439b-bb7b-e55ce0d8aa01 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:51.862848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be01a3-fde1-439b-bb7b-e55ce0d8aa01 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:51.864520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be01a3-fde1-439b-bb7b-e55ce0d8aa01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:20:51.865213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be01a3-fde1-439b-bb7b-e55ce0d8aa01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:51.867260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be01a3-fde1-439b-bb7b-e55ce0d8aa01 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:51.880423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 14/27] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:20:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:20:51.881298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5399be-ed4a-448f-9ca0-875ecd422cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:51.882211 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5399be-ed4a-448f-9ca0-875ecd422cab None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:51.882703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5399be-ed4a-448f-9ca0-875ecd422cab None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:51.883169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5399be-ed4a-448f-9ca0-875ecd422cab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:51.885147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d11e0a-a4c6-4b5a-b24d-dca93942f2e3 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:51.885802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d11e0a-a4c6-4b5a-b24d-dca93942f2e3 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:51.887420 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d11e0a-a4c6-4b5a-b24d-dca93942f2e3 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:51.889776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 15/28] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:20:51.895673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d11e0a-a4c6-4b5a-b24d-dca93942f2e3 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:51.907717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5399be-ed4a-448f-9ca0-875ecd422cab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:51.919805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5399be-ed4a-448f-9ca0-875ecd422cab None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-ff5399be-ed4a-448f-9ca0-875ecd422cab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:20:51.955331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:20:51.956219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:20:51.957031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:20:51.957031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:20:51.957031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:20:51.957031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:20:51.959641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 14/29] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:20:51.961049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2c641-cc26-4e1a-8455-de57cf01adf3 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:51.961554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2c641-cc26-4e1a-8455-de57cf01adf3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:51.962043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2c641-cc26-4e1a-8455-de57cf01adf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:51.986592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:51.999300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-23d2c641-cc26-4e1a-8455-de57cf01adf3 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:52.000921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-23d2c641-cc26-4e1a-8455-de57cf01adf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:52.134837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 15/30] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:20:52.135614 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2c641-cc26-4e1a-8455-de57cf01adf3 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:52.194384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:52.201416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d11e0a-a4c6-4b5a-b24d-dca93942f2e3 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:52.239195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7abeb-1cd5-4aa3-9dea-f4ad455ff8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:52.240014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7abeb-1cd5-4aa3-9dea-f4ad455ff8e6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:52.240306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7abeb-1cd5-4aa3-9dea-f4ad455ff8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:52.240591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7abeb-1cd5-4aa3-9dea-f4ad455ff8e6 None admin] PATH_INFO: `/v3/users/ec8458e10b6c4d69ba0ad246e923be31` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:52.241796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7abeb-1cd5-4aa3-9dea-f4ad455ff8e6 None admin] RBAC: Authorizing `identity:get_user(user_id=ec8458e10b6c4d69ba0ad246e923be31)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:52.243282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7abeb-1cd5-4aa3-9dea-f4ad455ff8e6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:52.249516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 16/31] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:20:51 2022] GET /identity/v3/users/ec8458e10b6c4d69ba0ad246e923be31 => generated 247 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:52.250549 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7abeb-1cd5-4aa3-9dea-f4ad455ff8e6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:52.270996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2c641-cc26-4e1a-8455-de57cf01adf3 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:52.312163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4f5cb-d476-4867-82ca-26908b3471c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:52.313075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4f5cb-d476-4867-82ca-26908b3471c5 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:52.313494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4f5cb-d476-4867-82ca-26908b3471c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:52.313981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4f5cb-d476-4867-82ca-26908b3471c5 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:52.314944 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4f5cb-d476-4867-82ca-26908b3471c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:20:52.315380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4f5cb-d476-4867-82ca-26908b3471c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:52.317643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4f5cb-d476-4867-82ca-26908b3471c5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:52.320645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:52.331900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 16/32] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:52.346803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16fa3c-7af6-4286-b617-f47e1f895af6 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:52.346803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16fa3c-7af6-4286-b617-f47e1f895af6 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:52.346803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16fa3c-7af6-4286-b617-f47e1f895af6 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:52.346803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 17/33] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:20:52.346803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16fa3c-7af6-4286-b617-f47e1f895af6 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:52.372885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7abeb-1cd5-4aa3-9dea-f4ad455ff8e6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:52.411610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbd226-8af8-4688-b16c-12a24e5a3ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:52.412156 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd226-8af8-4688-b16c-12a24e5a3ec8 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:52.412447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd226-8af8-4688-b16c-12a24e5a3ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:52.412783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd226-8af8-4688-b16c-12a24e5a3ec8 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:52.413522 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbd226-8af8-4688-b16c-12a24e5a3ec8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:52.414991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbd226-8af8-4688-b16c-12a24e5a3ec8 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:52.453000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16fa3c-7af6-4286-b617-f47e1f895af6 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:52.491427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe278e-c66d-43cf-8022-c4edc20a9927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:52.492085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe278e-c66d-43cf-8022-c4edc20a9927 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:52.492421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe278e-c66d-43cf-8022-c4edc20a9927 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:52.492702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe278e-c66d-43cf-8022-c4edc20a9927 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:52.493998 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe278e-c66d-43cf-8022-c4edc20a9927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:52.495618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe278e-c66d-43cf-8022-c4edc20a9927 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:52.502473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 18/34] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:20:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:52.502473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe278e-c66d-43cf-8022-c4edc20a9927 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:52.598437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:52.630284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 17/35] 10.222.0.73 () {60 vars in 1131 bytes} [Thu Aug 4 21:20:52 2022] POST /identity/v3/roles => generated 219 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:20:52.635652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76783aca-6e83-4351-921a-cad72b34c0fa None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:52.635934 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76783aca-6e83-4351-921a-cad72b34c0fa None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:52.636222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76783aca-6e83-4351-921a-cad72b34c0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:52.644902 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe278e-c66d-43cf-8022-c4edc20a9927 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:52.685466 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-76783aca-6e83-4351-921a-cad72b34c0fa None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:52.687090 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-76783aca-6e83-4351-921a-cad72b34c0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:52.690656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a0356-957a-4171-a5ec-afb89c856e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:52.691686 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a0356-957a-4171-a5ec-afb89c856e83 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:52.693007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a0356-957a-4171-a5ec-afb89c856e83 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:52.693614 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a0356-957a-4171-a5ec-afb89c856e83 None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:52.694571 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a0356-957a-4171-a5ec-afb89c856e83 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:52.696479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a0356-957a-4171-a5ec-afb89c856e83 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:52.701344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:52.769506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 18/36] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:20:52.770372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76783aca-6e83-4351-921a-cad72b34c0fa None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:52.891774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76783aca-6e83-4351-921a-cad72b34c0fa None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:52.930507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493369b0-e352-45df-907f-9fedba088b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:52.931183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493369b0-e352-45df-907f-9fedba088b64 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:52.931469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493369b0-e352-45df-907f-9fedba088b64 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:52.931748 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493369b0-e352-45df-907f-9fedba088b64 None admin] PATH_INFO: `/v3/domains/default/users/ec8458e10b6c4d69ba0ad246e923be31/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:52.933765 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493369b0-e352-45df-907f-9fedba088b64 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ec8458e10b6c4d69ba0ad246e923be31, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:52.934549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 19/37] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:20:52 2022] POST /identity/v3/projects => generated 301 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:20:52.935368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493369b0-e352-45df-907f-9fedba088b64 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:52.935823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bdbb8-9a0b-40ea-8df1-d18597b2237c None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:52.936073 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bdbb8-9a0b-40ea-8df1-d18597b2237c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:52.936424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bdbb8-9a0b-40ea-8df1-d18597b2237c None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:52.937962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 20/38] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:20:52.939158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabc60d-caa6-498a-9fa8-7e1b066bb6e6 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:52.939506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabc60d-caa6-498a-9fa8-7e1b066bb6e6 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:52.939791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabc60d-caa6-498a-9fa8-7e1b066bb6e6 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:52.941045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 21/39] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:20:52.941806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabc60d-caa6-498a-9fa8-7e1b066bb6e6 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:53.026875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 19/40] 10.222.0.73 () {58 vars in 1361 bytes} [Thu Aug 4 21:20:52 2022] PUT /identity/v3/domains/default/users/ec8458e10b6c4d69ba0ad246e923be31/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:20:53.028288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a387cf-57c1-4245-8d4d-7522df2059bb None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:53.028676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a387cf-57c1-4245-8d4d-7522df2059bb None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:53.028970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a387cf-57c1-4245-8d4d-7522df2059bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:53.043420 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:53.049949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-25a387cf-57c1-4245-8d4d-7522df2059bb None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:53.052253 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-25a387cf-57c1-4245-8d4d-7522df2059bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:53.076974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabc60d-caa6-498a-9fa8-7e1b066bb6e6 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:53.104708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:53.131177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be448ba-9b54-4eba-9b36-11351276fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:53.131878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be448ba-9b54-4eba-9b36-11351276fc5b None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:53.132206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be448ba-9b54-4eba-9b36-11351276fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:53.132489 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be448ba-9b54-4eba-9b36-11351276fc5b None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:53.133229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be448ba-9b54-4eba-9b36-11351276fc5b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:53.134870 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be448ba-9b54-4eba-9b36-11351276fc5b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:53.139925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 20/41] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:20:53.141533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c6867-3e8f-463e-a82c-4c4b6663d6c0 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:53.141533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c6867-3e8f-463e-a82c-4c4b6663d6c0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:53.141696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c6867-3e8f-463e-a82c-4c4b6663d6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:53.165894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-754c6867-3e8f-463e-a82c-4c4b6663d6c0 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:53.167355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-754c6867-3e8f-463e-a82c-4c4b6663d6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:53.243003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 21/42] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:20:53.244469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c6867-3e8f-463e-a82c-4c4b6663d6c0 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:53.250238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 22/43] 10.222.0.73 () {60 vars in 1131 bytes} [Thu Aug 4 21:20:52 2022] POST /identity/v3/roles => generated 225 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:20:53.274314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be448ba-9b54-4eba-9b36-11351276fc5b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:53.338366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:53.388469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c6867-3e8f-463e-a82c-4c4b6663d6c0 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:53.422641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be448ba-9b54-4eba-9b36-11351276fc5b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:53.433321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b492fd-370f-489e-bd05-7557cdc2fc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:53.434137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b492fd-370f-489e-bd05-7557cdc2fc57 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:53.434531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b492fd-370f-489e-bd05-7557cdc2fc57 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:53.434908 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b492fd-370f-489e-bd05-7557cdc2fc57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:53.436172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b492fd-370f-489e-bd05-7557cdc2fc57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:53.437804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b492fd-370f-489e-bd05-7557cdc2fc57 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:53.439518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 22/44] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:20:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:53.447013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b492fd-370f-489e-bd05-7557cdc2fc57 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:53.469809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883dc696-0fe4-4b1b-9179-f07bdb9b4b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:53.470635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dc696-0fe4-4b1b-9179-f07bdb9b4b40 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:53.471021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dc696-0fe4-4b1b-9179-f07bdb9b4b40 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:53.471393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dc696-0fe4-4b1b-9179-f07bdb9b4b40 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:53.472215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883dc696-0fe4-4b1b-9179-f07bdb9b4b40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:53.474403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883dc696-0fe4-4b1b-9179-f07bdb9b4b40 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:53.543501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 23/45] 10.222.0.73 () {60 vars in 1131 bytes} [Thu Aug 4 21:20:53 2022] POST /identity/v3/roles => generated 223 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:20:53.566492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b492fd-370f-489e-bd05-7557cdc2fc57 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:53.608504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c804ef9-415f-4649-81c1-165659177b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:53.609719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c804ef9-415f-4649-81c1-165659177b47 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:53.610035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c804ef9-415f-4649-81c1-165659177b47 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:53.610371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c804ef9-415f-4649-81c1-165659177b47 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:53.611296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c804ef9-415f-4649-81c1-165659177b47 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:53.612844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c804ef9-415f-4649-81c1-165659177b47 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:53.619484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:53.708012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 23/46] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:20:53 2022] POST /identity/v3/projects => generated 312 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:20:55.088538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd8810-581a-44f7-8265-42c8a5e63821 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:55.089060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd8810-581a-44f7-8265-42c8a5e63821 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:55.089144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd8810-581a-44f7-8265-42c8a5e63821 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:55.090482 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 24/47] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:20:55.095947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cd7fd-fc7c-4f89-8836-0f0b7fe1895c None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:55.096359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cd7fd-fc7c-4f89-8836-0f0b7fe1895c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:55.096541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cd7fd-fc7c-4f89-8836-0f0b7fe1895c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:55.115630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-3a7cd7fd-fc7c-4f89-8836-0f0b7fe1895c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:55.116825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3a7cd7fd-fc7c-4f89-8836-0f0b7fe1895c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:55.185325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 24/48] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:20:55.210915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd8810-581a-44f7-8265-42c8a5e63821 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:55.248426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:55.313543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd8810-581a-44f7-8265-42c8a5e63821 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:55.348401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38974004-33d3-44a2-b799-2e2b1f58cbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:55.348963 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38974004-33d3-44a2-b799-2e2b1f58cbf0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:55.349250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38974004-33d3-44a2-b799-2e2b1f58cbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:55.349529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38974004-33d3-44a2-b799-2e2b1f58cbf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:55.359924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38974004-33d3-44a2-b799-2e2b1f58cbf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-38974004-33d3-44a2-b799-2e2b1f58cbf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:20:55.365357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:20:55.366202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:20:55.367134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 25/49] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:20:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:20:55.372520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cd7fd-fc7c-4f89-8836-0f0b7fe1895c None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:55.470368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cd7fd-fc7c-4f89-8836-0f0b7fe1895c None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:55.486695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:55.505233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69361e43-b602-4cb2-8b48-e8dae84a17ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:55.505850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69361e43-b602-4cb2-8b48-e8dae84a17ee None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:55.506170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69361e43-b602-4cb2-8b48-e8dae84a17ee None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:55.506449 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69361e43-b602-4cb2-8b48-e8dae84a17ee None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:55.507170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69361e43-b602-4cb2-8b48-e8dae84a17ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:20:55.507481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69361e43-b602-4cb2-8b48-e8dae84a17ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:55.508870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69361e43-b602-4cb2-8b48-e8dae84a17ee None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:55.516886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 25/50] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:20:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:55.522853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38974004-33d3-44a2-b799-2e2b1f58cbf0 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:55.619598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:55.625174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38974004-33d3-44a2-b799-2e2b1f58cbf0 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:55.659016 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435b8d9-0d61-452f-a452-31220661aafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:55.659664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435b8d9-0d61-452f-a452-31220661aafa None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:55.659953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435b8d9-0d61-452f-a452-31220661aafa None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:55.660257 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435b8d9-0d61-452f-a452-31220661aafa None admin] PATH_INFO: `/v3/users/ec8458e10b6c4d69ba0ad246e923be31` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:55.661960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435b8d9-0d61-452f-a452-31220661aafa None admin] RBAC: Authorizing `identity:get_user(user_id=ec8458e10b6c4d69ba0ad246e923be31)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:55.663458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435b8d9-0d61-452f-a452-31220661aafa None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:55.665066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 26/51] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:20:55 2022] GET /identity/v3/users/ec8458e10b6c4d69ba0ad246e923be31 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:55.670674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69361e43-b602-4cb2-8b48-e8dae84a17ee None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:55.705367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002d374-e75e-498c-85ba-a5bc5541f0d4 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:55.705659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002d374-e75e-498c-85ba-a5bc5541f0d4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:55.705948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002d374-e75e-498c-85ba-a5bc5541f0d4 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:55.707955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 27/52] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:20:55.712980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70132e2-871f-4763-9b76-8cf48a236387 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:55.713266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70132e2-871f-4763-9b76-8cf48a236387 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:55.713542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70132e2-871f-4763-9b76-8cf48a236387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:55.732999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-e70132e2-871f-4763-9b76-8cf48a236387 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:55.734792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e70132e2-871f-4763-9b76-8cf48a236387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:55.774459 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69361e43-b602-4cb2-8b48-e8dae84a17ee None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:55.804295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 28/53] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:20:55.810785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a8175-3875-4000-a3ce-38480a22fe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:55.811441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8175-3875-4000-a3ce-38480a22fe21 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:55.811730 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8175-3875-4000-a3ce-38480a22fe21 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:55.812015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8175-3875-4000-a3ce-38480a22fe21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:55.813259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a8175-3875-4000-a3ce-38480a22fe21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:55.814752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a8175-3875-4000-a3ce-38480a22fe21 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:55.816474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 26/54] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:20:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:55.822718 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70132e2-871f-4763-9b76-8cf48a236387 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:55.826736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a8175-3875-4000-a3ce-38480a22fe21 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:55.873401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:55.920208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:55.927378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a8175-3875-4000-a3ce-38480a22fe21 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:55.928548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70132e2-871f-4763-9b76-8cf48a236387 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:55.960368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34e6f6-1cfb-4bea-abcc-f6ba925d8bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:55.961007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e6f6-1cfb-4bea-abcc-f6ba925d8bd7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:55.961290 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e6f6-1cfb-4bea-abcc-f6ba925d8bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:55.961561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e6f6-1cfb-4bea-abcc-f6ba925d8bd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:55.962726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34e6f6-1cfb-4bea-abcc-f6ba925d8bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:55.963136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d74d8-5c22-4ea6-a4d3-1167bf98fa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:55.963765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d74d8-5c22-4ea6-a4d3-1167bf98fa7b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:55.964055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d74d8-5c22-4ea6-a4d3-1167bf98fa7b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:55.964154 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34e6f6-1cfb-4bea-abcc-f6ba925d8bd7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:55.964336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d74d8-5c22-4ea6-a4d3-1167bf98fa7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:55.965123 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d74d8-5c22-4ea6-a4d3-1167bf98fa7b None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.domain.id=default, user.id=ec8458e10b6c4d69ba0ad246e923be31) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:20:55.965438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d74d8-5c22-4ea6-a4d3-1167bf98fa7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:55.965714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 27/55] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:20:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:20:55.966986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d74d8-5c22-4ea6-a4d3-1167bf98fa7b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:55.971256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34e6f6-1cfb-4bea-abcc-f6ba925d8bd7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:55.975997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 29/56] 10.222.0.73 () {56 vars in 1322 bytes} [Thu Aug 4 21:20:55 2022] GET /identity/v3/role_assignments?user.id=ec8458e10b6c4d69ba0ad246e923be31&scope.domain.id=default&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:56.081346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34e6f6-1cfb-4bea-abcc-f6ba925d8bd7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:56.099923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:56.121378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef3953-0ae4-403e-ab1f-c9cd5611ac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:56.122031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3953-0ae4-403e-ab1f-c9cd5611ac69 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:56.122330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3953-0ae4-403e-ab1f-c9cd5611ac69 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:56.122608 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3953-0ae4-403e-ab1f-c9cd5611ac69 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:56.123310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef3953-0ae4-403e-ab1f-c9cd5611ac69 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:56.124812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef3953-0ae4-403e-ab1f-c9cd5611ac69 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:56.272850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 28/57] 10.222.0.73 () {60 vars in 1140 bytes} [Thu Aug 4 21:20:55 2022] POST /identity/v3/projects => generated 298 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:20:58.296110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab3c8c-9c22-4342-a18a-cae119d9d186 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:58.296656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab3c8c-9c22-4342-a18a-cae119d9d186 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:58.296656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab3c8c-9c22-4342-a18a-cae119d9d186 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:58.298048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 30/58] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:20:58.303721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861b821-fbec-4152-808b-3eb2a2b07485 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:58.304117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861b821-fbec-4152-808b-3eb2a2b07485 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:58.304290 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861b821-fbec-4152-808b-3eb2a2b07485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:58.321822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-6861b821-fbec-4152-808b-3eb2a2b07485 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:20:58.323044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6861b821-fbec-4152-808b-3eb2a2b07485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:20:58.386451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 29/59] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:20:58.410728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab3c8c-9c22-4342-a18a-cae119d9d186 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:58.514291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab3c8c-9c22-4342-a18a-cae119d9d186 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:58.550426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12df5-0aba-45df-872b-552dafe62db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:58.551013 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12df5-0aba-45df-872b-552dafe62db7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:58.551300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12df5-0aba-45df-872b-552dafe62db7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:58.551658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12df5-0aba-45df-872b-552dafe62db7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:58.552822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12df5-0aba-45df-872b-552dafe62db7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:58.554322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12df5-0aba-45df-872b-552dafe62db7 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:58.555899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 31/60] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:20:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:20:58.561548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861b821-fbec-4152-808b-3eb2a2b07485 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:20:58.652681 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:20:58.658139 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861b821-fbec-4152-808b-3eb2a2b07485 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:20:58.690558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f314c3-7c17-4f7e-9017-a47591db8bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:20:58.691173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f314c3-7c17-4f7e-9017-a47591db8bec None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:20:58.691455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f314c3-7c17-4f7e-9017-a47591db8bec None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:20:58.691729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f314c3-7c17-4f7e-9017-a47591db8bec None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:20:58.693042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f314c3-7c17-4f7e-9017-a47591db8bec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:20:58.694833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f314c3-7c17-4f7e-9017-a47591db8bec None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:20:58.813726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 30/61] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:20:58 2022] POST /identity/v3/users => generated 275 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:01.999100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d35de6-c0bf-41aa-8de8-5ee9e2f489e5 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:01.999616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d35de6-c0bf-41aa-8de8-5ee9e2f489e5 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:01.999735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d35de6-c0bf-41aa-8de8-5ee9e2f489e5 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.001282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 32/62] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:02.007624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d01ea4-b7d8-45c7-b1e6-1d68ab83be5b None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.008088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d01ea4-b7d8-45c7-b1e6-1d68ab83be5b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.008180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d01ea4-b7d8-45c7-b1e6-1d68ab83be5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.039067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-11d01ea4-b7d8-45c7-b1e6-1d68ab83be5b None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:02.040282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-11d01ea4-b7d8-45c7-b1e6-1d68ab83be5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:02.144802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 31/63] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:02.191855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d35de6-c0bf-41aa-8de8-5ee9e2f489e5 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:02.237721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:02.311088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c6f06-1052-4c4e-a23e-0a4b0976e60a None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.311700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c6f06-1052-4c4e-a23e-0a4b0976e60a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.312183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c6f06-1052-4c4e-a23e-0a4b0976e60a None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.322123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 32/64] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:02.325143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235a893-a6a5-44ab-b5cc-ae713f725f30 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.325679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235a893-a6a5-44ab-b5cc-ae713f725f30 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.326192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235a893-a6a5-44ab-b5cc-ae713f725f30 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.327744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 33/65] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:21:02.333228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971772dd-1762-4f4c-afd7-cc8af8c604c1 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.333877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971772dd-1762-4f4c-afd7-cc8af8c604c1 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.334380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971772dd-1762-4f4c-afd7-cc8af8c604c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.339933 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d35de6-c0bf-41aa-8de8-5ee9e2f489e5 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:02.356353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-971772dd-1762-4f4c-afd7-cc8af8c604c1 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:02.358252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-971772dd-1762-4f4c-afd7-cc8af8c604c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:02.385040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2c4a4-71e5-4935-bc4a-951310322543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:02.385721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c4a4-71e5-4935-bc4a-951310322543 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.386028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c4a4-71e5-4935-bc4a-951310322543 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.386310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c4a4-71e5-4935-bc4a-951310322543 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.403751 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2c4a4-71e5-4935-bc4a-951310322543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-1dc2c4a4-71e5-4935-bc4a-951310322543 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:02.410095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:02.411638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:02.412574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 33/66] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:02.413408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532e1d8-5933-4224-9617-05a0153ab280 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.419977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532e1d8-5933-4224-9617-05a0153ab280 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.419977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532e1d8-5933-4224-9617-05a0153ab280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.440569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-8532e1d8-5933-4224-9617-05a0153ab280 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:02.443296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8532e1d8-5933-4224-9617-05a0153ab280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:02.458410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 34/67] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:02.461812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971772dd-1762-4f4c-afd7-cc8af8c604c1 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:02.486526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:02.563942 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 34/68] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:02.565205 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532e1d8-5933-4224-9617-05a0153ab280 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:02.591252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:02.609007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971772dd-1762-4f4c-afd7-cc8af8c604c1 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:02.651770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4366a95-5351-4b3a-8864-89eb63cf2731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:02.652406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4366a95-5351-4b3a-8864-89eb63cf2731 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.652693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4366a95-5351-4b3a-8864-89eb63cf2731 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.652970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4366a95-5351-4b3a-8864-89eb63cf2731 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.653694 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4366a95-5351-4b3a-8864-89eb63cf2731 None admin] RBAC: Adding query filter params (name=member) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:02.654040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4366a95-5351-4b3a-8864-89eb63cf2731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:02.655429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4366a95-5351-4b3a-8864-89eb63cf2731 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:02.664321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 35/69] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:02.665160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4366a95-5351-4b3a-8864-89eb63cf2731 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:02.689842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532e1d8-5933-4224-9617-05a0153ab280 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:02.729784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f4c8-f1ea-4cc9-ae43-9eb7c91cb4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:02.730562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f4c8-f1ea-4cc9-ae43-9eb7c91cb4fd None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.730846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f4c8-f1ea-4cc9-ae43-9eb7c91cb4fd None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.731118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f4c8-f1ea-4cc9-ae43-9eb7c91cb4fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.745012 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971f4c8-f1ea-4cc9-ae43-9eb7c91cb4fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:02.745874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-c971f4c8-f1ea-4cc9-ae43-9eb7c91cb4fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:02.750089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:02.751213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:02.751978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 35/70] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:21:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:02.752599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f477b-8d81-4a12-9903-b93ec382a27a None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.752760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f477b-8d81-4a12-9903-b93ec382a27a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.753083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f477b-8d81-4a12-9903-b93ec382a27a None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.754297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 36/71] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:02.755837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5c8da-b83d-4d61-9826-4acd0f6aa462 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.756124 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5c8da-b83d-4d61-9826-4acd0f6aa462 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.756402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5c8da-b83d-4d61-9826-4acd0f6aa462 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.758261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 37/72] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:02.761324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5c8da-b83d-4d61-9826-4acd0f6aa462 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:02.781399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4366a95-5351-4b3a-8864-89eb63cf2731 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:02.815522 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4b9d4-8103-4921-8c76-0655610e4b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:02.816062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4b9d4-8103-4921-8c76-0655610e4b57 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.816355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4b9d4-8103-4921-8c76-0655610e4b57 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.816637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4b9d4-8103-4921-8c76-0655610e4b57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.827821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4b9d4-8103-4921-8c76-0655610e4b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-66e4b9d4-8103-4921-8c76-0655610e4b57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:02.833820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:02.834727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:02.835474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 36/73] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:02.836185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4b9d4-8103-4921-8c76-0655610e4b57 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:02.842878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:02.868186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5c8da-b83d-4d61-9826-4acd0f6aa462 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:02.905038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d7180-97a0-43cd-9d39-ec700f074325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:02.905681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d7180-97a0-43cd-9d39-ec700f074325 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.905968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d7180-97a0-43cd-9d39-ec700f074325 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.906281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d7180-97a0-43cd-9d39-ec700f074325 None admin] PATH_INFO: `/v3/users/b1c1f798af854a34b553bedc48b56c28` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.920536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d7180-97a0-43cd-9d39-ec700f074325 None admin] RBAC: Authorizing `identity:get_user(user_id=b1c1f798af854a34b553bedc48b56c28)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:02.922202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d7180-97a0-43cd-9d39-ec700f074325 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:02.923918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 38/74] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:02 2022] GET /identity/v3/users/b1c1f798af854a34b553bedc48b56c28 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:02.925097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b7e81-8b39-4046-b645-84b3a1992146 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.925384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b7e81-8b39-4046-b645-84b3a1992146 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.925673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b7e81-8b39-4046-b645-84b3a1992146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.939091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4b9d4-8103-4921-8c76-0655610e4b57 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:02.946083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-be8b7e81-8b39-4046-b645-84b3a1992146 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:02.947381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-be8b7e81-8b39-4046-b645-84b3a1992146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:02.975465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c7a6e-4fae-45d8-a902-893568f9fd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:02.976097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c7a6e-4fae-45d8-a902-893568f9fd4d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.976384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c7a6e-4fae-45d8-a902-893568f9fd4d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.976659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c7a6e-4fae-45d8-a902-893568f9fd4d None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:02.977377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c7a6e-4fae-45d8-a902-893568f9fd4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:02.977692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c7a6e-4fae-45d8-a902-893568f9fd4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:02.979112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c7a6e-4fae-45d8-a902-893568f9fd4d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:02.980811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:02.989963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 37/75] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:02.991279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d0b33-7384-4178-a08a-1057ec7572c8 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:02.991572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d0b33-7384-4178-a08a-1057ec7572c8 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:02.991852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d0b33-7384-4178-a08a-1057ec7572c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:03.011407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-d95d0b33-7384-4178-a08a-1057ec7572c8 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:03.012658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d95d0b33-7384-4178-a08a-1057ec7572c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:03.031185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 39/76] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:03.032136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b7e81-8b39-4046-b645-84b3a1992146 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:03.082061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 38/77] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:03.082910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d0b33-7384-4178-a08a-1057ec7572c8 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:03.116485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:03.156168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b7e81-8b39-4046-b645-84b3a1992146 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:03.191095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbfc4c-1b27-4d47-b6aa-bde39718117f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:03.191868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbfc4c-1b27-4d47-b6aa-bde39718117f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:03.192713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbfc4c-1b27-4d47-b6aa-bde39718117f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:03.193383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbfc4c-1b27-4d47-b6aa-bde39718117f None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:03.193694 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbfc4c-1b27-4d47-b6aa-bde39718117f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:03.194046 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbfc4c-1b27-4d47-b6aa-bde39718117f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:03.195431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbfc4c-1b27-4d47-b6aa-bde39718117f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:03.199482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d0b33-7384-4178-a08a-1057ec7572c8 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:03.206969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 40/78] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:03.209019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbfc4c-1b27-4d47-b6aa-bde39718117f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:03.226678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:03.235567 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dde7a1-56a5-4a47-acfa-1d3e5e52ed11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:03.236200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dde7a1-56a5-4a47-acfa-1d3e5e52ed11 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:03.236486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dde7a1-56a5-4a47-acfa-1d3e5e52ed11 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:03.237302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dde7a1-56a5-4a47-acfa-1d3e5e52ed11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:03.309463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dde7a1-56a5-4a47-acfa-1d3e5e52ed11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:03.310986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dde7a1-56a5-4a47-acfa-1d3e5e52ed11 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:03.312092 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbfc4c-1b27-4d47-b6aa-bde39718117f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:03.337642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:03.346249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d048e-025a-4ae8-8b11-271a0551c0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:03.346906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d048e-025a-4ae8-8b11-271a0551c0e6 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:03.347186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d048e-025a-4ae8-8b11-271a0551c0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:03.351649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d048e-025a-4ae8-8b11-271a0551c0e6 None admin] PATH_INFO: `/v3/users/ec8458e10b6c4d69ba0ad246e923be31` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:03.351649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d048e-025a-4ae8-8b11-271a0551c0e6 None admin] RBAC: Authorizing `identity:get_user(user_id=ec8458e10b6c4d69ba0ad246e923be31)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:03.351649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d048e-025a-4ae8-8b11-271a0551c0e6 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:03.352333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 41/79] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:03 2022] GET /identity/v3/users/ec8458e10b6c4d69ba0ad246e923be31 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:03.353087 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d048e-025a-4ae8-8b11-271a0551c0e6 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:03.415559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 39/80] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:03.416412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dde7a1-56a5-4a47-acfa-1d3e5e52ed11 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:03.448523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:03.470334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d048e-025a-4ae8-8b11-271a0551c0e6 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:03.505966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e23fef-4477-4a21-aee5-f2d77eca5e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:03.506646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e23fef-4477-4a21-aee5-f2d77eca5e39 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:03.506932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e23fef-4477-4a21-aee5-f2d77eca5e39 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:03.507212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e23fef-4477-4a21-aee5-f2d77eca5e39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:03.517643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e23fef-4477-4a21-aee5-f2d77eca5e39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-37e23fef-4477-4a21-aee5-f2d77eca5e39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:03.523841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:03.524695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 42/81] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:03.529831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e23fef-4477-4a21-aee5-f2d77eca5e39 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:03.531981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dde7a1-56a5-4a47-acfa-1d3e5e52ed11 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:03.568623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1b4de-99a1-446c-88bf-7e640e16bb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:03.569278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1b4de-99a1-446c-88bf-7e640e16bb87 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:03.569564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1b4de-99a1-446c-88bf-7e640e16bb87 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:03.569843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1b4de-99a1-446c-88bf-7e640e16bb87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:03.571068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1b4de-99a1-446c-88bf-7e640e16bb87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:03.572588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1b4de-99a1-446c-88bf-7e640e16bb87 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:03.574139 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 40/82] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:03.574895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1b4de-99a1-446c-88bf-7e640e16bb87 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:03.585511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:03.630863 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e23fef-4477-4a21-aee5-f2d77eca5e39 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:03.664609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63227192-4839-4ba7-a966-6373e4e40ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:03.665258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63227192-4839-4ba7-a966-6373e4e40ebe None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:03.665541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63227192-4839-4ba7-a966-6373e4e40ebe None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:03.665820 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63227192-4839-4ba7-a966-6373e4e40ebe None admin] PATH_INFO: `/v3/users/b1c1f798af854a34b553bedc48b56c28` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:03.667032 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63227192-4839-4ba7-a966-6373e4e40ebe None admin] RBAC: Authorizing `identity:get_user(user_id=b1c1f798af854a34b553bedc48b56c28)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:03.668503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63227192-4839-4ba7-a966-6373e4e40ebe None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:03.670127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 43/83] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:03 2022] GET /identity/v3/users/b1c1f798af854a34b553bedc48b56c28 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:03.670250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:03.670893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63227192-4839-4ba7-a966-6373e4e40ebe None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:03.679237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1b4de-99a1-446c-88bf-7e640e16bb87 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:03.715720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568944c-e33d-48e9-a09c-56a476a40bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:03.716344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568944c-e33d-48e9-a09c-56a476a40bc1 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:03.716629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568944c-e33d-48e9-a09c-56a476a40bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:03.716903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568944c-e33d-48e9-a09c-56a476a40bc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:03.782093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63227192-4839-4ba7-a966-6373e4e40ebe None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:03.796929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568944c-e33d-48e9-a09c-56a476a40bc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:03.798457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568944c-e33d-48e9-a09c-56a476a40bc1 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:03.804844 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:03.816089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f493e-bb42-4c6d-b6a9-e589f29ef082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:03.816845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f493e-bb42-4c6d-b6a9-e589f29ef082 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:03.817973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f493e-bb42-4c6d-b6a9-e589f29ef082 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:03.818145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f493e-bb42-4c6d-b6a9-e589f29ef082 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:03.819250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f493e-bb42-4c6d-b6a9-e589f29ef082 None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:03.820724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f493e-bb42-4c6d-b6a9-e589f29ef082 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:03.822426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 44/84] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:03 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:03.823188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f493e-bb42-4c6d-b6a9-e589f29ef082 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:03.896869 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:03.904605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 41/85] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:03.918510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568944c-e33d-48e9-a09c-56a476a40bc1 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:03.931654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f493e-bb42-4c6d-b6a9-e589f29ef082 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:03.968147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9364a-f8b6-4999-abc6-8d2627b2ab7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:03.968770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9364a-f8b6-4999-abc6-8d2627b2ab7c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:03.969052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9364a-f8b6-4999-abc6-8d2627b2ab7c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:03.969325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9364a-f8b6-4999-abc6-8d2627b2ab7c None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:03.970061 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9364a-f8b6-4999-abc6-8d2627b2ab7c None admin] RBAC: Adding query filter params (name=member) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:03.970376 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9364a-f8b6-4999-abc6-8d2627b2ab7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:03.971772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9364a-f8b6-4999-abc6-8d2627b2ab7c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:03.984072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 45/86] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:03.985173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9364a-f8b6-4999-abc6-8d2627b2ab7c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:04.020573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568944c-e33d-48e9-a09c-56a476a40bc1 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:04.039922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:04.063431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c442350-4b3c-43f7-9c1e-49b5461ccf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:04.064059 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c442350-4b3c-43f7-9c1e-49b5461ccf21 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:04.064356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c442350-4b3c-43f7-9c1e-49b5461ccf21 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:04.064634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c442350-4b3c-43f7-9c1e-49b5461ccf21 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:04.065322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c442350-4b3c-43f7-9c1e-49b5461ccf21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:04.066816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c442350-4b3c-43f7-9c1e-49b5461ccf21 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:04.089128 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9364a-f8b6-4999-abc6-8d2627b2ab7c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:04.123463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c2b0b-d7fd-45c5-9c34-eca161bd8a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:04.124094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c2b0b-d7fd-45c5-9c34-eca161bd8a0b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:04.124380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c2b0b-d7fd-45c5-9c34-eca161bd8a0b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:04.124657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c2b0b-d7fd-45c5-9c34-eca161bd8a0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:04.173426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 42/87] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:21:03 2022] POST /identity/v3/users => generated 289 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:04.174233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c442350-4b3c-43f7-9c1e-49b5461ccf21 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:04.198129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c2b0b-d7fd-45c5-9c34-eca161bd8a0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:04.199695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c2b0b-d7fd-45c5-9c34-eca161bd8a0b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:04.245334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:04.265641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:04.303339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c442350-4b3c-43f7-9c1e-49b5461ccf21 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:04.320903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 46/88] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:04.321692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c2b0b-d7fd-45c5-9c34-eca161bd8a0b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:04.347521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee751e0f-e5f5-4929-9b9e-5d71b7a2971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:04.348150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee751e0f-e5f5-4929-9b9e-5d71b7a2971d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:04.349068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee751e0f-e5f5-4929-9b9e-5d71b7a2971d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:04.349347 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee751e0f-e5f5-4929-9b9e-5d71b7a2971d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:04.351118 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee751e0f-e5f5-4929-9b9e-5d71b7a2971d None admin] RBAC: Adding query filter params (role.id=36291c067d1949cfae0d704585f036c8, scope.project.id=6d126b37c8ae4111b880ecd5592237b8, user.id=b1c1f798af854a34b553bedc48b56c28) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:04.351118 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee751e0f-e5f5-4929-9b9e-5d71b7a2971d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:04.351935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee751e0f-e5f5-4929-9b9e-5d71b7a2971d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:04.358520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 43/89] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:04 2022] GET /identity/v3/role_assignments?user.id=b1c1f798af854a34b553bedc48b56c28&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=36291c067d1949cfae0d704585f036c8 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:04.362056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee751e0f-e5f5-4929-9b9e-5d71b7a2971d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:04.465428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c2b0b-d7fd-45c5-9c34-eca161bd8a0b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:04.509895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee751e0f-e5f5-4929-9b9e-5d71b7a2971d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:04.527970 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da544b84-dad9-49f7-8d16-b0a801f91537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:04.527970 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da544b84-dad9-49f7-8d16-b0a801f91537 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:04.528250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da544b84-dad9-49f7-8d16-b0a801f91537 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:04.528491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da544b84-dad9-49f7-8d16-b0a801f91537 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:04.529903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da544b84-dad9-49f7-8d16-b0a801f91537 None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:04.531809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da544b84-dad9-49f7-8d16-b0a801f91537 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:04.533705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 47/90] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:04 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:04.535105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da544b84-dad9-49f7-8d16-b0a801f91537 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:04.535483 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:04.567110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:04.568640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa3d38-cf2f-4fda-8070-7af5b40951b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:04.569533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3d38-cf2f-4fda-8070-7af5b40951b9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:04.570093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3d38-cf2f-4fda-8070-7af5b40951b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:04.570575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3d38-cf2f-4fda-8070-7af5b40951b9 None admin] PATH_INFO: `/v3/users/b1c1f798af854a34b553bedc48b56c28` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:04.572935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa3d38-cf2f-4fda-8070-7af5b40951b9 None admin] RBAC: Authorizing `identity:get_user(user_id=b1c1f798af854a34b553bedc48b56c28)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:04.575383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa3d38-cf2f-4fda-8070-7af5b40951b9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:04.577260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 44/91] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:04 2022] GET /identity/v3/users/b1c1f798af854a34b553bedc48b56c28 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:04.581195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa3d38-cf2f-4fda-8070-7af5b40951b9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:04.700357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da544b84-dad9-49f7-8d16-b0a801f91537 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:04.700855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa3d38-cf2f-4fda-8070-7af5b40951b9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:04.750597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6042b92-5ac1-402e-8604-3449274d8c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:04.751283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6042b92-5ac1-402e-8604-3449274d8c4d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:04.751580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6042b92-5ac1-402e-8604-3449274d8c4d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:04.751862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6042b92-5ac1-402e-8604-3449274d8c4d None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:04.753041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6042b92-5ac1-402e-8604-3449274d8c4d None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:04.755123 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6042b92-5ac1-402e-8604-3449274d8c4d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:04.756956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 48/92] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:04 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:04.757741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6042b92-5ac1-402e-8604-3449274d8c4d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:04.757848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe76d2-0760-4a39-a57d-be8db83e087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:04.758519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe76d2-0760-4a39-a57d-be8db83e087d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:04.758864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe76d2-0760-4a39-a57d-be8db83e087d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:04.759152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe76d2-0760-4a39-a57d-be8db83e087d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:04.759961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe76d2-0760-4a39-a57d-be8db83e087d None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=6d126b37c8ae4111b880ecd5592237b8, user.id=ec8458e10b6c4d69ba0ad246e923be31) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:04.760279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe76d2-0760-4a39-a57d-be8db83e087d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:04.761942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe76d2-0760-4a39-a57d-be8db83e087d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:04.775838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 45/93] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:04 2022] GET /identity/v3/role_assignments?user.id=ec8458e10b6c4d69ba0ad246e923be31&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:04.779819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe76d2-0760-4a39-a57d-be8db83e087d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:04.872730 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:04.938205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe76d2-0760-4a39-a57d-be8db83e087d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:04.978114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:04.992729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6042b92-5ac1-402e-8604-3449274d8c4d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:05.037281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53fe2c-0d72-477b-9e75-4a9e95fbd051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:05.038302 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53fe2c-0d72-477b-9e75-4a9e95fbd051 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:05.039054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53fe2c-0d72-477b-9e75-4a9e95fbd051 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:05.039534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53fe2c-0d72-477b-9e75-4a9e95fbd051 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:05.061853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27047f55-e93f-4615-bfef-d2c5a9b9cbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:05.062507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27047f55-e93f-4615-bfef-d2c5a9b9cbc9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:05.062795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27047f55-e93f-4615-bfef-d2c5a9b9cbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:05.063079 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27047f55-e93f-4615-bfef-d2c5a9b9cbc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:05.063880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27047f55-e93f-4615-bfef-d2c5a9b9cbc9 None admin] RBAC: Adding query filter params (role.id=677f038038444f229f95fc80c3166f24, scope.project.id=6d126b37c8ae4111b880ecd5592237b8, user.id=b1c1f798af854a34b553bedc48b56c28) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:05.064197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27047f55-e93f-4615-bfef-d2c5a9b9cbc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:05.065639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27047f55-e93f-4615-bfef-d2c5a9b9cbc9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:05.079948 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 46/94] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:04 2022] GET /identity/v3/role_assignments?user.id=b1c1f798af854a34b553bedc48b56c28&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=677f038038444f229f95fc80c3166f24 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:05.132072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53fe2c-0d72-477b-9e75-4a9e95fbd051 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:05.134380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53fe2c-0d72-477b-9e75-4a9e95fbd051 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:05.249456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:05.266842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 49/95] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:05.273170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27047f55-e93f-4615-bfef-d2c5a9b9cbc9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:05.407197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27047f55-e93f-4615-bfef-d2c5a9b9cbc9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:05.419305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:05.444583 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95067d14-ae20-4fc7-86f9-bbc20f9e547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:05.445610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95067d14-ae20-4fc7-86f9-bbc20f9e547f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:05.446176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95067d14-ae20-4fc7-86f9-bbc20f9e547f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:05.446650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95067d14-ae20-4fc7-86f9-bbc20f9e547f None admin] PATH_INFO: `/v3/projects/343b2ab677c34d43b5445509c6080b85` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:05.448120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95067d14-ae20-4fc7-86f9-bbc20f9e547f None admin] RBAC: Authorizing `identity:get_project(project_id=343b2ab677c34d43b5445509c6080b85)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:05.449839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95067d14-ae20-4fc7-86f9-bbc20f9e547f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:05.451729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 47/96] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:05 2022] GET /identity/v3/projects/343b2ab677c34d43b5445509c6080b85 => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:05.460098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53fe2c-0d72-477b-9e75-4a9e95fbd051 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:05.607349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53fe2c-0d72-477b-9e75-4a9e95fbd051 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:05.658889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519f7fb-2ea8-4dd0-acec-7dd208ddbd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:05.659862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519f7fb-2ea8-4dd0-acec-7dd208ddbd1d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:05.660360 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519f7fb-2ea8-4dd0-acec-7dd208ddbd1d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:05.661209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519f7fb-2ea8-4dd0-acec-7dd208ddbd1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:05.662223 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519f7fb-2ea8-4dd0-acec-7dd208ddbd1d None admin] RBAC: Adding query filter params (role.id=36291c067d1949cfae0d704585f036c8, scope.project.id=343b2ab677c34d43b5445509c6080b85, user.id=b1c1f798af854a34b553bedc48b56c28) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:05.662738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519f7fb-2ea8-4dd0-acec-7dd208ddbd1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:05.664467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519f7fb-2ea8-4dd0-acec-7dd208ddbd1d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:05.674287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 50/97] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:05 2022] GET /identity/v3/role_assignments?user.id=b1c1f798af854a34b553bedc48b56c28&scope.project.id=343b2ab677c34d43b5445509c6080b85&role.id=36291c067d1949cfae0d704585f036c8 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:06.994670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586da47-85f6-4abb-b3ee-381e715d68d1 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:06.995750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586da47-85f6-4abb-b3ee-381e715d68d1 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:06.996242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586da47-85f6-4abb-b3ee-381e715d68d1 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:06.997844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 48/98] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:07.003166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d92db-8572-4d8c-b90b-c7864d119dfb None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:07.003959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d92db-8572-4d8c-b90b-c7864d119dfb None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:07.004951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d92db-8572-4d8c-b90b-c7864d119dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:07.026071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-5f2d92db-8572-4d8c-b90b-c7864d119dfb None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:07.027633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5f2d92db-8572-4d8c-b90b-c7864d119dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:07.080026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:07.100782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 51/99] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:07.135264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586da47-85f6-4abb-b3ee-381e715d68d1 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:07.250174 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586da47-85f6-4abb-b3ee-381e715d68d1 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:07.268647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e8845-0b3e-45ac-808a-fcc923bc840e None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:07.268956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e8845-0b3e-45ac-808a-fcc923bc840e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:07.269212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e8845-0b3e-45ac-808a-fcc923bc840e None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:07.270713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 52/100] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:07.275731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5183430-5b6a-4074-a772-0990a61b39d7 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:07.276080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5183430-5b6a-4074-a772-0990a61b39d7 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:07.276368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5183430-5b6a-4074-a772-0990a61b39d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:07.307560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-f5183430-5b6a-4074-a772-0990a61b39d7 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:07.308782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f5183430-5b6a-4074-a772-0990a61b39d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:07.318117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea6b9f-04cb-4493-ab89-0370f0c71548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:07.318785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea6b9f-04cb-4493-ab89-0370f0c71548 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:07.319067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea6b9f-04cb-4493-ab89-0370f0c71548 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:07.319336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea6b9f-04cb-4493-ab89-0370f0c71548 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:07.345556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea6b9f-04cb-4493-ab89-0370f0c71548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-a8ea6b9f-04cb-4493-ab89-0370f0c71548 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:07.351553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:07.352412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:07.353209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 49/101] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:07.360500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea6b9f-04cb-4493-ab89-0370f0c71548 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:07.384591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 53/102] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:07.422100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5183430-5b6a-4074-a772-0990a61b39d7 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:07.437314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:07.525451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea6b9f-04cb-4493-ab89-0370f0c71548 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:07.562036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc42e54-deea-4d8f-a264-5a58e2e993cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:07.562036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc42e54-deea-4d8f-a264-5a58e2e993cc None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:07.562036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc42e54-deea-4d8f-a264-5a58e2e993cc None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:07.563090 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc42e54-deea-4d8f-a264-5a58e2e993cc None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:07.564459 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc42e54-deea-4d8f-a264-5a58e2e993cc None admin] RBAC: Adding query filter params (name=member) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:07.566043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc42e54-deea-4d8f-a264-5a58e2e993cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:07.568146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc42e54-deea-4d8f-a264-5a58e2e993cc None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:07.583956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 50/103] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:07.585438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd83c5-ea03-4e57-b1d3-c09b2f5b6792 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:07.588564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd83c5-ea03-4e57-b1d3-c09b2f5b6792 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:07.588564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd83c5-ea03-4e57-b1d3-c09b2f5b6792 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:07.588564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 51/104] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:07.596536 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788098a5-8cc4-4e93-bdd2-bd6fc2b63718 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:07.596838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788098a5-8cc4-4e93-bdd2-bd6fc2b63718 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:07.599463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788098a5-8cc4-4e93-bdd2-bd6fc2b63718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:07.632281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-788098a5-8cc4-4e93-bdd2-bd6fc2b63718 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:07.633553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-788098a5-8cc4-4e93-bdd2-bd6fc2b63718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:07.641284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5183430-5b6a-4074-a772-0990a61b39d7 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:07.670388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:07.701367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24390c9-74d4-4086-9e2c-faecf8954c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:07.702039 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24390c9-74d4-4086-9e2c-faecf8954c51 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:07.702324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24390c9-74d4-4086-9e2c-faecf8954c51 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:07.702598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24390c9-74d4-4086-9e2c-faecf8954c51 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:07.727592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24390c9-74d4-4086-9e2c-faecf8954c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-a24390c9-74d4-4086-9e2c-faecf8954c51 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:07.733795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:07.735009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:07.735853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 54/105] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:07.737690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24390c9-74d4-4086-9e2c-faecf8954c51 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:07.769752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 52/106] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:07.771586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788098a5-8cc4-4e93-bdd2-bd6fc2b63718 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:07.843994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:07.911758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24390c9-74d4-4086-9e2c-faecf8954c51 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:07.932836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788098a5-8cc4-4e93-bdd2-bd6fc2b63718 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:07.961549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2cd86-59dc-4e6e-9ae2-7fcf2ce6831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:07.962789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2cd86-59dc-4e6e-9ae2-7fcf2ce6831f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:07.963661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2cd86-59dc-4e6e-9ae2-7fcf2ce6831f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:07.964393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2cd86-59dc-4e6e-9ae2-7fcf2ce6831f None admin] PATH_INFO: `/v3/users/b1c1f798af854a34b553bedc48b56c28` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:07.966181 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2cd86-59dc-4e6e-9ae2-7fcf2ce6831f None admin] RBAC: Authorizing `identity:get_user(user_id=b1c1f798af854a34b553bedc48b56c28)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:07.968512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2cd86-59dc-4e6e-9ae2-7fcf2ce6831f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:07.976122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 55/107] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:07 2022] GET /identity/v3/users/b1c1f798af854a34b553bedc48b56c28 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:07.976122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2cd86-59dc-4e6e-9ae2-7fcf2ce6831f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:07.976284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20fc8e-7011-485a-b41e-2650e5a11d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:07.976284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20fc8e-7011-485a-b41e-2650e5a11d42 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:07.976284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20fc8e-7011-485a-b41e-2650e5a11d42 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:07.976284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20fc8e-7011-485a-b41e-2650e5a11d42 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:07.976544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20fc8e-7011-485a-b41e-2650e5a11d42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:07.976544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20fc8e-7011-485a-b41e-2650e5a11d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:07.976544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20fc8e-7011-485a-b41e-2650e5a11d42 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:07.991047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 53/108] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:07.992671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20909f9-1725-42ee-9b26-ab0db6e5aca6 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:07.993128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20909f9-1725-42ee-9b26-ab0db6e5aca6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:07.993570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20909f9-1725-42ee-9b26-ab0db6e5aca6 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:07.995039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 54/109] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:07.996593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20909f9-1725-42ee-9b26-ab0db6e5aca6 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:08.036197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:08.099068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:08.103232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2cd86-59dc-4e6e-9ae2-7fcf2ce6831f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:08.110053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20909f9-1725-42ee-9b26-ab0db6e5aca6 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:08.142391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d55448-a4aa-49ae-995f-1a1866d2932a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:08.143141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55448-a4aa-49ae-995f-1a1866d2932a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.143383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55448-a4aa-49ae-995f-1a1866d2932a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.143662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55448-a4aa-49ae-995f-1a1866d2932a None admin] PATH_INFO: `/v3/domains/default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.144872 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d55448-a4aa-49ae-995f-1a1866d2932a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:08.146570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d33a7b-0ea1-406e-8810-3d30067d3039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:08.147020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d55448-a4aa-49ae-995f-1a1866d2932a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:08.147206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d33a7b-0ea1-406e-8810-3d30067d3039 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.147495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d33a7b-0ea1-406e-8810-3d30067d3039 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.147772 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d33a7b-0ea1-406e-8810-3d30067d3039 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.148832 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 56/110] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:08.149528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d55448-a4aa-49ae-995f-1a1866d2932a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:08.153750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d33a7b-0ea1-406e-8810-3d30067d3039 None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:08.155272 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d33a7b-0ea1-406e-8810-3d30067d3039 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:08.157116 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 55/111] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:07 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:08.163517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1234d1-0708-49a2-b86d-dcba55337150 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.163858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1234d1-0708-49a2-b86d-dcba55337150 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.164138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1234d1-0708-49a2-b86d-dcba55337150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.183588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-4c1234d1-0708-49a2-b86d-dcba55337150 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:08.184790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4c1234d1-0708-49a2-b86d-dcba55337150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:08.266031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 56/112] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:08.266213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d55448-a4aa-49ae-995f-1a1866d2932a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:08.266328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef45a2-d29f-40fa-8789-11cc9ebd596e None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.266328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef45a2-d29f-40fa-8789-11cc9ebd596e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.266328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef45a2-d29f-40fa-8789-11cc9ebd596e None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.266328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 57/113] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:08.266328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef45a2-d29f-40fa-8789-11cc9ebd596e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:08.279994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:08.307981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9440a-c821-42cc-be82-29d16b09ab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:08.308554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9440a-c821-42cc-be82-29d16b09ab5a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.308841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9440a-c821-42cc-be82-29d16b09ab5a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.309121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9440a-c821-42cc-be82-29d16b09ab5a None admin] PATH_INFO: `/v3/users/36027a139e6e4266b13fc58e6db2c379` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.324101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9440a-c821-42cc-be82-29d16b09ab5a None admin] RBAC: Authorizing `identity:get_user(user_id=36027a139e6e4266b13fc58e6db2c379)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:08.325591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9440a-c821-42cc-be82-29d16b09ab5a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:08.331712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 57/114] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:08 2022] GET /identity/v3/users/36027a139e6e4266b13fc58e6db2c379 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:08.331712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9440a-c821-42cc-be82-29d16b09ab5a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:08.368200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:08.378987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef45a2-d29f-40fa-8789-11cc9ebd596e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:08.413915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4204406-028d-497c-89df-16fe03bfdb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:08.414595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204406-028d-497c-89df-16fe03bfdb24 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.414854 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204406-028d-497c-89df-16fe03bfdb24 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.415138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204406-028d-497c-89df-16fe03bfdb24 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.415826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4204406-028d-497c-89df-16fe03bfdb24 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:08.417273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4204406-028d-497c-89df-16fe03bfdb24 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:08.440829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9440a-c821-42cc-be82-29d16b09ab5a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:08.475948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23b1fa-7cb1-4c90-a0f9-3cb1a2b5c068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:08.476626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b1fa-7cb1-4c90-a0f9-3cb1a2b5c068 None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.476915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b1fa-7cb1-4c90-a0f9-3cb1a2b5c068 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.477203 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b1fa-7cb1-4c90-a0f9-3cb1a2b5c068 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8/users/b1c1f798af854a34b553bedc48b56c28/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.479206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23b1fa-7cb1-4c90-a0f9-3cb1a2b5c068 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d126b37c8ae4111b880ecd5592237b8, user_id=b1c1f798af854a34b553bedc48b56c28, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:08.480649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23b1fa-7cb1-4c90-a0f9-3cb1a2b5c068 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:08.506473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 58/115] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:21:08 2022] POST /identity/v3/projects => generated 302 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:08.514831 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b1b63-2185-4d21-8b26-ad04f0ddcfc9 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.515164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b1b63-2185-4d21-8b26-ad04f0ddcfc9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.515437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b1b63-2185-4d21-8b26-ad04f0ddcfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.537572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-be1b1b63-2185-4d21-8b26-ad04f0ddcfc9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:08.538851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-be1b1b63-2185-4d21-8b26-ad04f0ddcfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:08.577347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 58/116] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:21:08 2022] PUT /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8/users/b1c1f798af854a34b553bedc48b56c28/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:08.578202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23b1fa-7cb1-4c90-a0f9-3cb1a2b5c068 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:08.611430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:08.643277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 59/117] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:08.644229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b1b63-2185-4d21-8b26-ad04f0ddcfc9 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:08.723792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23b1fa-7cb1-4c90-a0f9-3cb1a2b5c068 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:08.739139 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:08.770751 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528391b-ae04-4b2b-81fb-3be9266e0814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:08.772242 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528391b-ae04-4b2b-81fb-3be9266e0814 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.772836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528391b-ae04-4b2b-81fb-3be9266e0814 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.773119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528391b-ae04-4b2b-81fb-3be9266e0814 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.777331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b1b63-2185-4d21-8b26-ad04f0ddcfc9 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:08.785030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528391b-ae04-4b2b-81fb-3be9266e0814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-9528391b-ae04-4b2b-81fb-3be9266e0814 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:08.790752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:08.791675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:08.792611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 59/118] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:08.793409 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528391b-ae04-4b2b-81fb-3be9266e0814 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:08.821269 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67413623-d9da-42e0-8d86-c345a6e290f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:08.821963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67413623-d9da-42e0-8d86-c345a6e290f7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.822276 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67413623-d9da-42e0-8d86-c345a6e290f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.822556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67413623-d9da-42e0-8d86-c345a6e290f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.903720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:08.915677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67413623-d9da-42e0-8d86-c345a6e290f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:08.917276 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67413623-d9da-42e0-8d86-c345a6e290f7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:08.922381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528391b-ae04-4b2b-81fb-3be9266e0814 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:08.970096 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435439d-2fa2-4962-8c0a-6a1403a4661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:08.970760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435439d-2fa2-4962-8c0a-6a1403a4661d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.971055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435439d-2fa2-4962-8c0a-6a1403a4661d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.971468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435439d-2fa2-4962-8c0a-6a1403a4661d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.985616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435439d-2fa2-4962-8c0a-6a1403a4661d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-7435439d-2fa2-4962-8c0a-6a1403a4661d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:08.993262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:08.994248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:08.995111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 60/119] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:21:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:08.995740 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67253381-59bb-4ce0-999c-4d488becc0e6 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:08.995957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67253381-59bb-4ce0-999c-4d488becc0e6 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:08.998335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67253381-59bb-4ce0-999c-4d488becc0e6 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:08.999900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 61/120] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:09.000753 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67253381-59bb-4ce0-999c-4d488becc0e6 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:09.002835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:09.038981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 60/121] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:09.040226 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67413623-d9da-42e0-8d86-c345a6e290f7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:09.150879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67253381-59bb-4ce0-999c-4d488becc0e6 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:09.155132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67413623-d9da-42e0-8d86-c345a6e290f7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:09.189355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd805d0f-ca1a-461f-828b-76feab8bf761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:09.189981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd805d0f-ca1a-461f-828b-76feab8bf761 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:09.190294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd805d0f-ca1a-461f-828b-76feab8bf761 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:09.190570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd805d0f-ca1a-461f-828b-76feab8bf761 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:09.191283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd805d0f-ca1a-461f-828b-76feab8bf761 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:09.191837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd805d0f-ca1a-461f-828b-76feab8bf761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:09.193521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd805d0f-ca1a-461f-828b-76feab8bf761 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:09.201921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 62/122] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:09.203164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db4c1-410a-42c7-aa40-9343b65c5379 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:09.203452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db4c1-410a-42c7-aa40-9343b65c5379 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:09.203734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db4c1-410a-42c7-aa40-9343b65c5379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:09.213177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f67c74-da91-4a51-8a78-91ebffa75fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:09.213835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f67c74-da91-4a51-8a78-91ebffa75fc4 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:09.213943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f67c74-da91-4a51-8a78-91ebffa75fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:09.214906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f67c74-da91-4a51-8a78-91ebffa75fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:09.214906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f67c74-da91-4a51-8a78-91ebffa75fc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:09.216704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f67c74-da91-4a51-8a78-91ebffa75fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:09.216704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f67c74-da91-4a51-8a78-91ebffa75fc4 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:09.225193 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 61/123] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:09.225431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-de8db4c1-410a-42c7-aa40-9343b65c5379 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:09.226213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f67c74-da91-4a51-8a78-91ebffa75fc4 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:09.226952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-de8db4c1-410a-42c7-aa40-9343b65c5379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:09.230343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:09.269273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:09.302144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 63/124] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:09.306780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8db4c1-410a-42c7-aa40-9343b65c5379 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:09.351911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f67c74-da91-4a51-8a78-91ebffa75fc4 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:09.400340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409adf3-982e-46aa-95fa-e18458926ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:09.401298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409adf3-982e-46aa-95fa-e18458926ed1 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:09.401763 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409adf3-982e-46aa-95fa-e18458926ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:09.402283 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409adf3-982e-46aa-95fa-e18458926ed1 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:09.403728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409adf3-982e-46aa-95fa-e18458926ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:09.406045 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409adf3-982e-46aa-95fa-e18458926ed1 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:09.408843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 62/125] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:09 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:09.415056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409adf3-982e-46aa-95fa-e18458926ed1 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:09.438396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8db4c1-410a-42c7-aa40-9343b65c5379 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:09.480739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f690d2-6ab9-4553-bc0d-95d195b02ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:09.482428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f690d2-6ab9-4553-bc0d-95d195b02ec7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:09.482919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f690d2-6ab9-4553-bc0d-95d195b02ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:09.483661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f690d2-6ab9-4553-bc0d-95d195b02ec7 None admin] PATH_INFO: `/v3/users/b1c1f798af854a34b553bedc48b56c28` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:09.485939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f690d2-6ab9-4553-bc0d-95d195b02ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=b1c1f798af854a34b553bedc48b56c28)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:09.488575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f690d2-6ab9-4553-bc0d-95d195b02ec7 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:09.490983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 64/126] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:09 2022] GET /identity/v3/users/b1c1f798af854a34b553bedc48b56c28 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:09.497039 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f690d2-6ab9-4553-bc0d-95d195b02ec7 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:09.507724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:09.537469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409adf3-982e-46aa-95fa-e18458926ed1 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:09.539989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:09.583856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b568ba9-96d7-4c4b-b4be-fabad829a8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:09.584783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b568ba9-96d7-4c4b-b4be-fabad829a8f5 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:09.585317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b568ba9-96d7-4c4b-b4be-fabad829a8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:09.585794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b568ba9-96d7-4c4b-b4be-fabad829a8f5 None admin] PATH_INFO: `/v3/users/ec8458e10b6c4d69ba0ad246e923be31` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:09.587295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b568ba9-96d7-4c4b-b4be-fabad829a8f5 None admin] RBAC: Authorizing `identity:get_user(user_id=ec8458e10b6c4d69ba0ad246e923be31)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:09.588933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b568ba9-96d7-4c4b-b4be-fabad829a8f5 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:09.590780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 63/127] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:09 2022] GET /identity/v3/users/ec8458e10b6c4d69ba0ad246e923be31 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:09.591801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b568ba9-96d7-4c4b-b4be-fabad829a8f5 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:09.631843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f690d2-6ab9-4553-bc0d-95d195b02ec7 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:09.674809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c126f6-92af-4997-a4c5-bba07c1ef9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:09.675517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c126f6-92af-4997-a4c5-bba07c1ef9c4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:09.675801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c126f6-92af-4997-a4c5-bba07c1ef9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:09.676080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c126f6-92af-4997-a4c5-bba07c1ef9c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:09.690641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c126f6-92af-4997-a4c5-bba07c1ef9c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:09.698924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b568ba9-96d7-4c4b-b4be-fabad829a8f5 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-b6c126f6-92af-4997-a4c5-bba07c1ef9c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:09.699478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:09.700332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:09.701530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 65/128] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:09.706460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c126f6-92af-4997-a4c5-bba07c1ef9c4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:09.744385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa5076-6ad4-4592-a700-c7e68c255251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:09.745534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5076-6ad4-4592-a700-c7e68c255251 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:09.746076 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5076-6ad4-4592-a700-c7e68c255251 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:09.746555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5076-6ad4-4592-a700-c7e68c255251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:09.747660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa5076-6ad4-4592-a700-c7e68c255251 None admin] RBAC: Adding query filter params (role.id=2ca1bca713f745ea9b40dc20cb5fa2a1, scope.project.id=6d126b37c8ae4111b880ecd5592237b8, user.id=36027a139e6e4266b13fc58e6db2c379) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:09.748202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa5076-6ad4-4592-a700-c7e68c255251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:09.749833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa5076-6ad4-4592-a700-c7e68c255251 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:09.762144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 64/129] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:09 2022] GET /identity/v3/role_assignments?user.id=36027a139e6e4266b13fc58e6db2c379&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=2ca1bca713f745ea9b40dc20cb5fa2a1 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:09.763504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa5076-6ad4-4592-a700-c7e68c255251 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:09.797003 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:09.846967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:09.850375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c126f6-92af-4997-a4c5-bba07c1ef9c4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:09.898767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead7c30-a107-4365-9a37-9495eaa24dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:09.899439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead7c30-a107-4365-9a37-9495eaa24dc0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:09.900022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead7c30-a107-4365-9a37-9495eaa24dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:09.900310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead7c30-a107-4365-9a37-9495eaa24dc0 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:09.902322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead7c30-a107-4365-9a37-9495eaa24dc0 None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:09.903833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead7c30-a107-4365-9a37-9495eaa24dc0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:09.906444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 66/130] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:09 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:09.911712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead7c30-a107-4365-9a37-9495eaa24dc0 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:09.989411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa5076-6ad4-4592-a700-c7e68c255251 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:10.026312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6014cf-624d-4fdc-b75c-2a1d9145e38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:10.026922 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6014cf-624d-4fdc-b75c-2a1d9145e38f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:10.027142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6014cf-624d-4fdc-b75c-2a1d9145e38f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:10.027367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6014cf-624d-4fdc-b75c-2a1d9145e38f None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:10.028546 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6014cf-624d-4fdc-b75c-2a1d9145e38f None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:10.030030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6014cf-624d-4fdc-b75c-2a1d9145e38f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:10.031698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 65/131] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:09 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:10.032433 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6014cf-624d-4fdc-b75c-2a1d9145e38f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:10.038069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead7c30-a107-4365-9a37-9495eaa24dc0 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:10.083106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcd115-704b-4ab5-8cca-19d71662c4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:10.085064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcd115-704b-4ab5-8cca-19d71662c4a7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:10.086302 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcd115-704b-4ab5-8cca-19d71662c4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:10.086823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcd115-704b-4ab5-8cca-19d71662c4a7 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:10.088379 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcd115-704b-4ab5-8cca-19d71662c4a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:10.089543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcd115-704b-4ab5-8cca-19d71662c4a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:10.091920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcd115-704b-4ab5-8cca-19d71662c4a7 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:10.094439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:10.112705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 67/132] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:10.114507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcd115-704b-4ab5-8cca-19d71662c4a7 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:10.238518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6014cf-624d-4fdc-b75c-2a1d9145e38f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:10.243486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:10.314514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcd115-704b-4ab5-8cca-19d71662c4a7 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:10.315877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b3b24-2516-44f0-a9c2-0ee098627d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:10.316890 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b3b24-2516-44f0-a9c2-0ee098627d73 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:10.317606 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b3b24-2516-44f0-a9c2-0ee098627d73 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:10.319081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b3b24-2516-44f0-a9c2-0ee098627d73 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8/users/b1c1f798af854a34b553bedc48b56c28/roles/677f038038444f229f95fc80c3166f24` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:10.326873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b3b24-2516-44f0-a9c2-0ee098627d73 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d126b37c8ae4111b880ecd5592237b8, user_id=b1c1f798af854a34b553bedc48b56c28, role_id=677f038038444f229f95fc80c3166f24)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:10.328633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b3b24-2516-44f0-a9c2-0ee098627d73 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:10.366564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7faae14-be7b-4295-920b-33a0a87d6a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:10.367246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7faae14-be7b-4295-920b-33a0a87d6a5d None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:10.367531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7faae14-be7b-4295-920b-33a0a87d6a5d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:10.368916 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7faae14-be7b-4295-920b-33a0a87d6a5d None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8/users/ec8458e10b6c4d69ba0ad246e923be31/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:10.370215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7faae14-be7b-4295-920b-33a0a87d6a5d None admin] RBAC: Authorizing `identity:create_grant(project_id=6d126b37c8ae4111b880ecd5592237b8, user_id=ec8458e10b6c4d69ba0ad246e923be31, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:10.371716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7faae14-be7b-4295-920b-33a0a87d6a5d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:10.421192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 66/133] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:21:10 2022] PUT /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8/users/b1c1f798af854a34b553bedc48b56c28/roles/677f038038444f229f95fc80c3166f24 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:10.422200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b3b24-2516-44f0-a9c2-0ee098627d73 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:10.456651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 68/134] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:21:10 2022] PUT /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8/users/ec8458e10b6c4d69ba0ad246e923be31/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:10.551734 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b3b24-2516-44f0-a9c2-0ee098627d73 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:10.591499 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659d2fc-a55c-4e62-a813-7de36abaf7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:10.592181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659d2fc-a55c-4e62-a813-7de36abaf7a9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:10.592461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659d2fc-a55c-4e62-a813-7de36abaf7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:10.592742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659d2fc-a55c-4e62-a813-7de36abaf7a9 None admin] PATH_INFO: `/v3/users/b1c1f798af854a34b553bedc48b56c28` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:10.594144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659d2fc-a55c-4e62-a813-7de36abaf7a9 None admin] RBAC: Authorizing `identity:get_user(user_id=b1c1f798af854a34b553bedc48b56c28)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:10.595661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659d2fc-a55c-4e62-a813-7de36abaf7a9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:10.597425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 67/135] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:10 2022] GET /identity/v3/users/b1c1f798af854a34b553bedc48b56c28 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:10.603902 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7faae14-be7b-4295-920b-33a0a87d6a5d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:10.674000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:10.729313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7faae14-be7b-4295-920b-33a0a87d6a5d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:10.777094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a755c8f-bf54-41ad-a9e7-584db81f54b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:10.777683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755c8f-bf54-41ad-a9e7-584db81f54b3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:10.777974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755c8f-bf54-41ad-a9e7-584db81f54b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:10.778278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755c8f-bf54-41ad-a9e7-584db81f54b3 None admin] PATH_INFO: `/v3/projects/343b2ab677c34d43b5445509c6080b85` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:10.779472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a755c8f-bf54-41ad-a9e7-584db81f54b3 None admin] RBAC: Authorizing `identity:get_project(project_id=343b2ab677c34d43b5445509c6080b85)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:10.780949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a755c8f-bf54-41ad-a9e7-584db81f54b3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:10.782972 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 69/136] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:10 2022] GET /identity/v3/projects/343b2ab677c34d43b5445509c6080b85 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:10.790461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659d2fc-a55c-4e62-a813-7de36abaf7a9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:10.824792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:10.912396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659d2fc-a55c-4e62-a813-7de36abaf7a9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:10.961680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f873bf-b05f-4a93-af11-ae5880d2248c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:10.962639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f873bf-b05f-4a93-af11-ae5880d2248c None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:10.962953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f873bf-b05f-4a93-af11-ae5880d2248c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:10.963395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f873bf-b05f-4a93-af11-ae5880d2248c None admin] PATH_INFO: `/v3/projects/343b2ab677c34d43b5445509c6080b85/users/b1c1f798af854a34b553bedc48b56c28/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:10.965410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f873bf-b05f-4a93-af11-ae5880d2248c None admin] RBAC: Authorizing `identity:create_grant(project_id=343b2ab677c34d43b5445509c6080b85, user_id=b1c1f798af854a34b553bedc48b56c28, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:10.966883 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f873bf-b05f-4a93-af11-ae5880d2248c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:11.001871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d098b1-d069-4be4-984a-c313fd240390 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:11.002191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d098b1-d069-4be4-984a-c313fd240390 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:11.002479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d098b1-d069-4be4-984a-c313fd240390 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:11.004076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 70/137] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:11.019867 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c44f1-acb3-4953-8435-d314de45563f None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:11.020358 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c44f1-acb3-4953-8435-d314de45563f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:11.020646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c44f1-acb3-4953-8435-d314de45563f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:11.048377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-785c44f1-acb3-4953-8435-d314de45563f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:11.048377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-785c44f1-acb3-4953-8435-d314de45563f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:11.074554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 68/138] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:21:10 2022] PUT /identity/v3/projects/343b2ab677c34d43b5445509c6080b85/users/b1c1f798af854a34b553bedc48b56c28/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:11.076649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de428e-4b8c-4503-a46d-3cc16b4504f6 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:11.076964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de428e-4b8c-4503-a46d-3cc16b4504f6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:11.077247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de428e-4b8c-4503-a46d-3cc16b4504f6 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:11.081075 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 69/139] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:11.087233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9cedf-26b8-4fe9-b8c4-dfbc6b888d87 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:11.088724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9cedf-26b8-4fe9-b8c4-dfbc6b888d87 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:11.088724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9cedf-26b8-4fe9-b8c4-dfbc6b888d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:11.122442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-38a9cedf-26b8-4fe9-b8c4-dfbc6b888d87 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:11.129005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-38a9cedf-26b8-4fe9-b8c4-dfbc6b888d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:11.188934 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 71/140] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:11.219812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 70/141] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:11.240323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c44f1-acb3-4953-8435-d314de45563f None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:11.253173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9cedf-26b8-4fe9-b8c4-dfbc6b888d87 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:11.295926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:11.355974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:11.421967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9cedf-26b8-4fe9-b8c4-dfbc6b888d87 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:11.459829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c44f1-acb3-4953-8435-d314de45563f None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:11.468818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a15a10-3fe1-4705-9b23-1f230c94db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:11.469853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15a10-3fe1-4705-9b23-1f230c94db41 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:11.470376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15a10-3fe1-4705-9b23-1f230c94db41 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:11.470828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15a10-3fe1-4705-9b23-1f230c94db41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:11.482800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a15a10-3fe1-4705-9b23-1f230c94db41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-f7a15a10-3fe1-4705-9b23-1f230c94db41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:11.489540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:11.490425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:11.491196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:11.493620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 71/142] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:11.499754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a15a10-3fe1-4705-9b23-1f230c94db41 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:11.562722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194210a-0055-4271-ad5e-fbfb76347ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:11.563392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194210a-0055-4271-ad5e-fbfb76347ef4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:11.563689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194210a-0055-4271-ad5e-fbfb76347ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:11.563973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194210a-0055-4271-ad5e-fbfb76347ef4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:11.565159 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194210a-0055-4271-ad5e-fbfb76347ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:11.566665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194210a-0055-4271-ad5e-fbfb76347ef4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:11.569177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 72/143] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:11.574935 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194210a-0055-4271-ad5e-fbfb76347ef4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:11.603744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:11.627941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a15a10-3fe1-4705-9b23-1f230c94db41 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:11.670654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ee650-d0de-4a2c-a5e0-057c96b99d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:11.671614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee650-d0de-4a2c-a5e0-057c96b99d9c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:11.672086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee650-d0de-4a2c-a5e0-057c96b99d9c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:11.672537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee650-d0de-4a2c-a5e0-057c96b99d9c None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:11.673423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ee650-d0de-4a2c-a5e0-057c96b99d9c None admin] RBAC: Adding query filter params (name=member) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:11.673936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ee650-d0de-4a2c-a5e0-057c96b99d9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:11.675510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ee650-d0de-4a2c-a5e0-057c96b99d9c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:11.686155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 72/144] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:11.693058 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ee650-d0de-4a2c-a5e0-057c96b99d9c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:11.812570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ee650-d0de-4a2c-a5e0-057c96b99d9c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:11.829169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194210a-0055-4271-ad5e-fbfb76347ef4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:11.878423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:11.888399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a5222-4188-47bd-8693-93c4cb53ff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:11.890729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a5222-4188-47bd-8693-93c4cb53ff0d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:11.891571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a5222-4188-47bd-8693-93c4cb53ff0d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:11.892053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a5222-4188-47bd-8693-93c4cb53ff0d None admin] PATH_INFO: `/v3/users/b1c1f798af854a34b553bedc48b56c28` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:11.895333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a5222-4188-47bd-8693-93c4cb53ff0d None admin] RBAC: Authorizing `identity:get_user(user_id=b1c1f798af854a34b553bedc48b56c28)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:11.899716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a5222-4188-47bd-8693-93c4cb53ff0d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:11.902747 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab674e4-f71f-4760-b7dc-74f0ec65f41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:11.903402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 73/145] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:11 2022] GET /identity/v3/users/b1c1f798af854a34b553bedc48b56c28 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:11.928843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab674e4-f71f-4760-b7dc-74f0ec65f41e None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:11.928843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab674e4-f71f-4760-b7dc-74f0ec65f41e None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:11.928843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab674e4-f71f-4760-b7dc-74f0ec65f41e None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:11.928843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab674e4-f71f-4760-b7dc-74f0ec65f41e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:11.928843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab674e4-f71f-4760-b7dc-74f0ec65f41e None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:11.929208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a5222-4188-47bd-8693-93c4cb53ff0d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:11.945795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:12.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 73/146] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:21:11 2022] POST /identity/v3/users => generated 283 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:21:12.066469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a5222-4188-47bd-8693-93c4cb53ff0d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:12.119114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15ba51-2593-4016-9ce7-b5e640f041d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:12.120552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15ba51-2593-4016-9ce7-b5e640f041d0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:12.121055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15ba51-2593-4016-9ce7-b5e640f041d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:12.121561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15ba51-2593-4016-9ce7-b5e640f041d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:12.204370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15ba51-2593-4016-9ce7-b5e640f041d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:12.206280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15ba51-2593-4016-9ce7-b5e640f041d0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:12.219011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:12.315867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 74/147] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:12.326042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab674e4-f71f-4760-b7dc-74f0ec65f41e None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:12.485765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab674e4-f71f-4760-b7dc-74f0ec65f41e None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:12.542351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df7728-7c1d-40d0-8ca6-d69cb42568bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:12.543011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df7728-7c1d-40d0-8ca6-d69cb42568bf None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:12.543357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df7728-7c1d-40d0-8ca6-d69cb42568bf None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:12.543643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df7728-7c1d-40d0-8ca6-d69cb42568bf None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:12.544831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df7728-7c1d-40d0-8ca6-d69cb42568bf None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:12.546324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df7728-7c1d-40d0-8ca6-d69cb42568bf None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:12.548150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 74/148] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:12 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:12.554114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15ba51-2593-4016-9ce7-b5e640f041d0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:12.695850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15ba51-2593-4016-9ce7-b5e640f041d0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:12.710082 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:12.742710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01d6ab-9cc1-4f3a-9529-da8273192e0e None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:12.742710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01d6ab-9cc1-4f3a-9529-da8273192e0e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:12.742710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01d6ab-9cc1-4f3a-9529-da8273192e0e None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:12.742710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 75/149] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:21:12.747670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1781e2-979f-4b16-abfc-331c070d92fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:12.748036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facced10-8f9e-46da-b04f-813e8db30067 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:12.748327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facced10-8f9e-46da-b04f-813e8db30067 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:12.748608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facced10-8f9e-46da-b04f-813e8db30067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:12.749201 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1781e2-979f-4b16-abfc-331c070d92fc None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:12.749670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1781e2-979f-4b16-abfc-331c070d92fc None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:12.750168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1781e2-979f-4b16-abfc-331c070d92fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:12.751226 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1781e2-979f-4b16-abfc-331c070d92fc None admin] RBAC: Adding query filter params (role.id=36291c067d1949cfae0d704585f036c8, scope.project.id=6d126b37c8ae4111b880ecd5592237b8, user.id=b1c1f798af854a34b553bedc48b56c28) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:12.751787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1781e2-979f-4b16-abfc-331c070d92fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:12.753396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1781e2-979f-4b16-abfc-331c070d92fc None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:12.760861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 75/150] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:12 2022] GET /identity/v3/role_assignments?user.id=b1c1f798af854a34b553bedc48b56c28&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=36291c067d1949cfae0d704585f036c8 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:12.777933 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-facced10-8f9e-46da-b04f-813e8db30067 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:12.779356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-facced10-8f9e-46da-b04f-813e8db30067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:12.860495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:12.888610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 76/151] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:12.927632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1781e2-979f-4b16-abfc-331c070d92fc None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:13.072225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1781e2-979f-4b16-abfc-331c070d92fc None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:13.112839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1de64e-ed86-4f77-bc62-f2cd1f519bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:13.113878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1de64e-ed86-4f77-bc62-f2cd1f519bc3 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:13.114388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1de64e-ed86-4f77-bc62-f2cd1f519bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:13.114864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1de64e-ed86-4f77-bc62-f2cd1f519bc3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:13.126792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1de64e-ed86-4f77-bc62-f2cd1f519bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-fc1de64e-ed86-4f77-bc62-f2cd1f519bc3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:13.134683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:13.135561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:13.136330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:13.138659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 76/152] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:13.147121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facced10-8f9e-46da-b04f-813e8db30067 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:13.262250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30702d-52b9-4d13-9d86-b0b05a7d53b8 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:13.262940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30702d-52b9-4d13-9d86-b0b05a7d53b8 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:13.263416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30702d-52b9-4d13-9d86-b0b05a7d53b8 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:13.265388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 77/153] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:13.271090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facced10-8f9e-46da-b04f-813e8db30067 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:13.271403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9077ab5-b547-4418-ab7b-3372873dcc04 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:13.271888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9077ab5-b547-4418-ab7b-3372873dcc04 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:13.272357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9077ab5-b547-4418-ab7b-3372873dcc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:13.295324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c9077ab5-b547-4418-ab7b-3372873dcc04 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:13.296849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c9077ab5-b547-4418-ab7b-3372873dcc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:13.329992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9237d-e72f-4454-8151-2f38e8d1cc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:13.330580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9237d-e72f-4454-8151-2f38e8d1cc36 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:13.330862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9237d-e72f-4454-8151-2f38e8d1cc36 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:13.331155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9237d-e72f-4454-8151-2f38e8d1cc36 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:13.331858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9237d-e72f-4454-8151-2f38e8d1cc36 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:13.332168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9237d-e72f-4454-8151-2f38e8d1cc36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:13.334223 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9237d-e72f-4454-8151-2f38e8d1cc36 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:13.343154 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:13.350131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 77/154] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:13.354231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9237d-e72f-4454-8151-2f38e8d1cc36 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:13.411781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 78/155] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:13.428791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:13.454364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9077ab5-b547-4418-ab7b-3372873dcc04 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:13.503183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9237d-e72f-4454-8151-2f38e8d1cc36 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:13.556018 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffad8a7-664e-49e9-bc77-136b19df8d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:13.556966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffad8a7-664e-49e9-bc77-136b19df8d5b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:13.557459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffad8a7-664e-49e9-bc77-136b19df8d5b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:13.557948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffad8a7-664e-49e9-bc77-136b19df8d5b None admin] PATH_INFO: `/v3/users/36027a139e6e4266b13fc58e6db2c379` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:13.559392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffad8a7-664e-49e9-bc77-136b19df8d5b None admin] RBAC: Authorizing `identity:get_user(user_id=36027a139e6e4266b13fc58e6db2c379)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:13.561174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffad8a7-664e-49e9-bc77-136b19df8d5b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:13.563030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 78/156] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:13 2022] GET /identity/v3/users/36027a139e6e4266b13fc58e6db2c379 => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:13.567441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbd73a-50b3-46a8-aa70-e76abc9e60ec None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:13.567989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbd73a-50b3-46a8-aa70-e76abc9e60ec None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:13.568505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbd73a-50b3-46a8-aa70-e76abc9e60ec None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:13.569837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 79/157] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:13.573403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbd73a-50b3-46a8-aa70-e76abc9e60ec None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:13.592678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9077ab5-b547-4418-ab7b-3372873dcc04 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:13.636395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d8fb6-c3dc-4c93-b22b-22f56a8b6d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:13.637047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d8fb6-c3dc-4c93-b22b-22f56a8b6d84 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:13.637390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d8fb6-c3dc-4c93-b22b-22f56a8b6d84 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:13.637670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d8fb6-c3dc-4c93-b22b-22f56a8b6d84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:13.649628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d8fb6-c3dc-4c93-b22b-22f56a8b6d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-5d9d8fb6-c3dc-4c93-b22b-22f56a8b6d84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:13.654671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:13.655560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:13.656336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 79/158] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:21:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:13.657900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7ce4a-5824-4065-9ab7-e9b38396f8e9 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:13.658210 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7ce4a-5824-4065-9ab7-e9b38396f8e9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:13.658510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7ce4a-5824-4065-9ab7-e9b38396f8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:13.679197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-69c7ce4a-5824-4065-9ab7-e9b38396f8e9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:13.680463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-69c7ce4a-5824-4065-9ab7-e9b38396f8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:13.696337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbd73a-50b3-46a8-aa70-e76abc9e60ec None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:13.704175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:13.724401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:13.736105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83cc3b6-a1c1-4c71-bfd7-c278fc6fad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:13.736926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cc3b6-a1c1-4c71-bfd7-c278fc6fad34 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:13.737514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cc3b6-a1c1-4c71-bfd7-c278fc6fad34 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:13.737796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cc3b6-a1c1-4c71-bfd7-c278fc6fad34 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:13.739595 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83cc3b6-a1c1-4c71-bfd7-c278fc6fad34 None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:13.741559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83cc3b6-a1c1-4c71-bfd7-c278fc6fad34 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:13.743637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 80/159] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:13 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:13.744589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83cc3b6-a1c1-4c71-bfd7-c278fc6fad34 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:13.762419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 80/160] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:13.763427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7ce4a-5824-4065-9ab7-e9b38396f8e9 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:13.953528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83cc3b6-a1c1-4c71-bfd7-c278fc6fad34 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:13.953794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7ce4a-5824-4065-9ab7-e9b38396f8e9 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:13.953887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd654f-0b72-4aa2-8d73-50e785d368d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:13.953887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd654f-0b72-4aa2-8d73-50e785d368d5 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:13.953887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd654f-0b72-4aa2-8d73-50e785d368d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:13.953887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd654f-0b72-4aa2-8d73-50e785d368d5 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:13.954197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd654f-0b72-4aa2-8d73-50e785d368d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:13.954197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd654f-0b72-4aa2-8d73-50e785d368d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:13.954197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd654f-0b72-4aa2-8d73-50e785d368d5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:13.954197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 81/161] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:13.954197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd654f-0b72-4aa2-8d73-50e785d368d5 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:13.990297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:14.032959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53606503-144a-4d2d-a5a2-a10c50b96ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:14.033661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53606503-144a-4d2d-a5a2-a10c50b96ad9 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:14.033993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53606503-144a-4d2d-a5a2-a10c50b96ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:14.034299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53606503-144a-4d2d-a5a2-a10c50b96ad9 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8/users/36027a139e6e4266b13fc58e6db2c379/roles/2ca1bca713f745ea9b40dc20cb5fa2a1` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:14.039387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53606503-144a-4d2d-a5a2-a10c50b96ad9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d126b37c8ae4111b880ecd5592237b8, user_id=36027a139e6e4266b13fc58e6db2c379, role_id=2ca1bca713f745ea9b40dc20cb5fa2a1)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:14.040849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53606503-144a-4d2d-a5a2-a10c50b96ad9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:14.063103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd654f-0b72-4aa2-8d73-50e785d368d5 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:14.100306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e45cf-fc5a-4844-a4f6-7d9ecbdc8c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:14.101020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e45cf-fc5a-4844-a4f6-7d9ecbdc8c16 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:14.101385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e45cf-fc5a-4844-a4f6-7d9ecbdc8c16 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:14.101716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e45cf-fc5a-4844-a4f6-7d9ecbdc8c16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:14.113107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e45cf-fc5a-4844-a4f6-7d9ecbdc8c16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-a20e45cf-fc5a-4844-a4f6-7d9ecbdc8c16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:14.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:14.119862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:14.120755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 82/162] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:14.121537 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e45cf-fc5a-4844-a4f6-7d9ecbdc8c16 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:14.227667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:14.230688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e45cf-fc5a-4844-a4f6-7d9ecbdc8c16 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:14.270602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47859c9e-c986-43fa-ab8e-e72d673f00e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:14.271544 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47859c9e-c986-43fa-ab8e-e72d673f00e8 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:14.271999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47859c9e-c986-43fa-ab8e-e72d673f00e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:14.272289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47859c9e-c986-43fa-ab8e-e72d673f00e8 None admin] PATH_INFO: `/v3/users/ec8458e10b6c4d69ba0ad246e923be31` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:14.273841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47859c9e-c986-43fa-ab8e-e72d673f00e8 None admin] RBAC: Authorizing `identity:get_user(user_id=ec8458e10b6c4d69ba0ad246e923be31)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:14.275511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47859c9e-c986-43fa-ab8e-e72d673f00e8 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:14.277935 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 83/163] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:14 2022] GET /identity/v3/users/ec8458e10b6c4d69ba0ad246e923be31 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:14.278741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47859c9e-c986-43fa-ab8e-e72d673f00e8 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:14.280192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 81/164] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:21:13 2022] PUT /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8/users/36027a139e6e4266b13fc58e6db2c379/roles/2ca1bca713f745ea9b40dc20cb5fa2a1 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:14.283798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53606503-144a-4d2d-a5a2-a10c50b96ad9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:14.322657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:14.410768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47859c9e-c986-43fa-ab8e-e72d673f00e8 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:14.455582 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f38b0-aee5-45bb-94fb-5731569d7dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:14.456219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f38b0-aee5-45bb-94fb-5731569d7dd9 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:14.456501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f38b0-aee5-45bb-94fb-5731569d7dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:14.456776 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f38b0-aee5-45bb-94fb-5731569d7dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:14.457664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f38b0-aee5-45bb-94fb-5731569d7dd9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:14.460985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f38b0-aee5-45bb-94fb-5731569d7dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:14.460985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f38b0-aee5-45bb-94fb-5731569d7dd9 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:14.469823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 84/165] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:14.476966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f38b0-aee5-45bb-94fb-5731569d7dd9 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:14.512334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53606503-144a-4d2d-a5a2-a10c50b96ad9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:14.537659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:14.594817 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3ae8b-2d29-4c45-b0a3-7dbb66c9f5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:14.595868 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3ae8b-2d29-4c45-b0a3-7dbb66c9f5a6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:14.596461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3ae8b-2d29-4c45-b0a3-7dbb66c9f5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:14.596933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3ae8b-2d29-4c45-b0a3-7dbb66c9f5a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:14.626171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f38b0-aee5-45bb-94fb-5731569d7dd9 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:14.670244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99a7ce-9fc0-41ab-a172-70a527d75690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:14.671176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99a7ce-9fc0-41ab-a172-70a527d75690 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:14.671478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99a7ce-9fc0-41ab-a172-70a527d75690 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:14.671758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99a7ce-9fc0-41ab-a172-70a527d75690 None admin] PATH_INFO: `/v3/users/b1c1f798af854a34b553bedc48b56c28` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:14.673914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99a7ce-9fc0-41ab-a172-70a527d75690 None admin] RBAC: Authorizing `identity:get_user(user_id=b1c1f798af854a34b553bedc48b56c28)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:14.675915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99a7ce-9fc0-41ab-a172-70a527d75690 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:14.678688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 85/166] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:14 2022] GET /identity/v3/users/b1c1f798af854a34b553bedc48b56c28 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:14.680418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e512403-6ed2-4785-ac23-468af9c501d9 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:14.680709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e512403-6ed2-4785-ac23-468af9c501d9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:14.680986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e512403-6ed2-4785-ac23-468af9c501d9 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:14.682512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 86/167] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:14.687154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e512403-6ed2-4785-ac23-468af9c501d9 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:14.698824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3ae8b-2d29-4c45-b0a3-7dbb66c9f5a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:14.701049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3ae8b-2d29-4c45-b0a3-7dbb66c9f5a6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:14.729863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:14.796029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:14.804248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e512403-6ed2-4785-ac23-468af9c501d9 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:14.817280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 82/168] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:14.818915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d93b1-5e2f-4a19-b984-b136ec8da2b7 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:14.819405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d93b1-5e2f-4a19-b984-b136ec8da2b7 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:14.819892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d93b1-5e2f-4a19-b984-b136ec8da2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:14.840194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-465d93b1-5e2f-4a19-b984-b136ec8da2b7 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:14.841255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24b6bf-5532-42aa-8a56-57ccf88f6bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:14.841886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24b6bf-5532-42aa-8a56-57ccf88f6bd9 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:14.842200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24b6bf-5532-42aa-8a56-57ccf88f6bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:14.842475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24b6bf-5532-42aa-8a56-57ccf88f6bd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:14.842870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-465d93b1-5e2f-4a19-b984-b136ec8da2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:14.915243 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24b6bf-5532-42aa-8a56-57ccf88f6bd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:14.916710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24b6bf-5532-42aa-8a56-57ccf88f6bd9 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:14.919530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 83/169] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:14.920598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d93b1-5e2f-4a19-b984-b136ec8da2b7 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:14.976289 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:15.028187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d93b1-5e2f-4a19-b984-b136ec8da2b7 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:15.040562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:15.048175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 87/170] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:15.048807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24b6bf-5532-42aa-8a56-57ccf88f6bd9 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:15.067213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d67a-447e-44c2-8311-09f6e33a3e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:15.068179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d67a-447e-44c2-8311-09f6e33a3e0f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:15.068645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d67a-447e-44c2-8311-09f6e33a3e0f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:15.069103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d67a-447e-44c2-8311-09f6e33a3e0f None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:15.070518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94d67a-447e-44c2-8311-09f6e33a3e0f None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:15.078031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94d67a-447e-44c2-8311-09f6e33a3e0f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:15.080146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 84/171] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:14 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:15.081206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d67a-447e-44c2-8311-09f6e33a3e0f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:15.157182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24b6bf-5532-42aa-8a56-57ccf88f6bd9 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:15.195351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1aca3d-a4a2-4b67-a4a8-46398f32ef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:15.196115 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aca3d-a4a2-4b67-a4a8-46398f32ef34 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:15.196469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aca3d-a4a2-4b67-a4a8-46398f32ef34 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:15.196817 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aca3d-a4a2-4b67-a4a8-46398f32ef34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:15.210056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1aca3d-a4a2-4b67-a4a8-46398f32ef34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-5b1aca3d-a4a2-4b67-a4a8-46398f32ef34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:15.213558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:15.214495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 88/172] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:15.215350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1aca3d-a4a2-4b67-a4a8-46398f32ef34 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:15.266969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d67a-447e-44c2-8311-09f6e33a3e0f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:15.298451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:15.319839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:15.342814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f9ac-79d3-4ea3-8518-f0c66d411e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:15.343876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f9ac-79d3-4ea3-8518-f0c66d411e4d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:15.344359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f9ac-79d3-4ea3-8518-f0c66d411e4d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:15.344826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f9ac-79d3-4ea3-8518-f0c66d411e4d None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:15.346397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216f9ac-79d3-4ea3-8518-f0c66d411e4d None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:15.348663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216f9ac-79d3-4ea3-8518-f0c66d411e4d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:15.354847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 85/173] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:15 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:15.360405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f9ac-79d3-4ea3-8518-f0c66d411e4d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:15.396701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1aca3d-a4a2-4b67-a4a8-46398f32ef34 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:15.456229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cef95d-73c3-46f0-9b91-8b832b2399e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:15.456892 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cef95d-73c3-46f0-9b91-8b832b2399e5 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:15.457243 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cef95d-73c3-46f0-9b91-8b832b2399e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:15.457715 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cef95d-73c3-46f0-9b91-8b832b2399e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:15.458605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cef95d-73c3-46f0-9b91-8b832b2399e5 None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=6d126b37c8ae4111b880ecd5592237b8, user.id=ec8458e10b6c4d69ba0ad246e923be31) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:15.458988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cef95d-73c3-46f0-9b91-8b832b2399e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:15.460569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cef95d-73c3-46f0-9b91-8b832b2399e5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:15.463608 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f9ac-79d3-4ea3-8518-f0c66d411e4d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:15.478317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 89/174] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:15 2022] GET /identity/v3/role_assignments?user.id=ec8458e10b6c4d69ba0ad246e923be31&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:15.478317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cef95d-73c3-46f0-9b91-8b832b2399e5 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:15.508349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2a57-c8a5-4208-9f9a-e2cc89f3f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:15.509363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2a57-c8a5-4208-9f9a-e2cc89f3f90c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:15.509850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2a57-c8a5-4208-9f9a-e2cc89f3f90c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:15.510346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2a57-c8a5-4208-9f9a-e2cc89f3f90c None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:15.511286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a2a57-c8a5-4208-9f9a-e2cc89f3f90c None admin] RBAC: Adding query filter params (name=member) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:15.511795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a2a57-c8a5-4208-9f9a-e2cc89f3f90c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:15.513562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a2a57-c8a5-4208-9f9a-e2cc89f3f90c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:15.535228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 86/175] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:15.543102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2a57-c8a5-4208-9f9a-e2cc89f3f90c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:15.610638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:15.702213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cef95d-73c3-46f0-9b91-8b832b2399e5 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:15.746412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:15.756807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2a57-c8a5-4208-9f9a-e2cc89f3f90c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:15.789903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daefa98-1e43-40ca-878c-961251ad088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:15.790566 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daefa98-1e43-40ca-878c-961251ad088b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:15.790855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daefa98-1e43-40ca-878c-961251ad088b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:15.791252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daefa98-1e43-40ca-878c-961251ad088b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:15.792063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daefa98-1e43-40ca-878c-961251ad088b None admin] RBAC: Adding query filter params (role.id=677f038038444f229f95fc80c3166f24, scope.project.id=6d126b37c8ae4111b880ecd5592237b8, user.id=b1c1f798af854a34b553bedc48b56c28) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:15.792375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daefa98-1e43-40ca-878c-961251ad088b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:15.793829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daefa98-1e43-40ca-878c-961251ad088b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:15.802899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 90/176] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:15 2022] GET /identity/v3/role_assignments?user.id=b1c1f798af854a34b553bedc48b56c28&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=677f038038444f229f95fc80c3166f24 => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:15.823736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63c70f-210a-4815-87da-dee8c579fd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:15.824392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63c70f-210a-4815-87da-dee8c579fd2c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:15.824676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63c70f-210a-4815-87da-dee8c579fd2c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:15.824952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63c70f-210a-4815-87da-dee8c579fd2c None admin] PATH_INFO: `/v3/users/b1c1f798af854a34b553bedc48b56c28` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:15.826217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63c70f-210a-4815-87da-dee8c579fd2c None admin] RBAC: Authorizing `identity:get_user(user_id=b1c1f798af854a34b553bedc48b56c28)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:15.827842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63c70f-210a-4815-87da-dee8c579fd2c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:15.829836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 87/177] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:15 2022] GET /identity/v3/users/b1c1f798af854a34b553bedc48b56c28 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:15.835719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daefa98-1e43-40ca-878c-961251ad088b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:15.955070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daefa98-1e43-40ca-878c-961251ad088b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:15.997804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee03d1-a089-40a8-b816-7024b3f07b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:15.998462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee03d1-a089-40a8-b816-7024b3f07b27 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:15.998747 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee03d1-a089-40a8-b816-7024b3f07b27 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:15.999082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee03d1-a089-40a8-b816-7024b3f07b27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:16.058976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:16.081015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee03d1-a089-40a8-b816-7024b3f07b27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:16.082603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee03d1-a089-40a8-b816-7024b3f07b27 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:16.216056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 91/178] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:16.221703 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63c70f-210a-4815-87da-dee8c579fd2c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:16.336746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63c70f-210a-4815-87da-dee8c579fd2c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:16.348683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:16.374549 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d58ccf-9a72-49e6-ad9d-085ae6847b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:16.375264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d58ccf-9a72-49e6-ad9d-085ae6847b00 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:16.375553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d58ccf-9a72-49e6-ad9d-085ae6847b00 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:16.375830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d58ccf-9a72-49e6-ad9d-085ae6847b00 None admin] PATH_INFO: `/v3/projects/343b2ab677c34d43b5445509c6080b85` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:16.377042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d58ccf-9a72-49e6-ad9d-085ae6847b00 None admin] RBAC: Authorizing `identity:get_project(project_id=343b2ab677c34d43b5445509c6080b85)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:16.378602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d58ccf-9a72-49e6-ad9d-085ae6847b00 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:16.380677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 88/179] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:16 2022] GET /identity/v3/projects/343b2ab677c34d43b5445509c6080b85 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:16.388099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee03d1-a089-40a8-b816-7024b3f07b27 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:16.485524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:16.512172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee03d1-a089-40a8-b816-7024b3f07b27 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:16.563171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4fefd7-eb9b-4e46-8714-f8eeff37814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:16.563798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fefd7-eb9b-4e46-8714-f8eeff37814b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:16.564074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fefd7-eb9b-4e46-8714-f8eeff37814b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:16.564406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fefd7-eb9b-4e46-8714-f8eeff37814b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:16.565202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4fefd7-eb9b-4e46-8714-f8eeff37814b None admin] RBAC: Adding query filter params (role.id=36291c067d1949cfae0d704585f036c8, scope.project.id=343b2ab677c34d43b5445509c6080b85, user.id=b1c1f798af854a34b553bedc48b56c28) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:16.565561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4fefd7-eb9b-4e46-8714-f8eeff37814b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:16.567310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4fefd7-eb9b-4e46-8714-f8eeff37814b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:16.573985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 92/180] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:16 2022] GET /identity/v3/role_assignments?user.id=b1c1f798af854a34b553bedc48b56c28&scope.project.id=343b2ab677c34d43b5445509c6080b85&role.id=36291c067d1949cfae0d704585f036c8 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:16.814609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78b3a9-fd1c-4efa-a0bf-f0d1b044d806 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:16.815262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78b3a9-fd1c-4efa-a0bf-f0d1b044d806 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:16.815767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78b3a9-fd1c-4efa-a0bf-f0d1b044d806 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:16.817355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 89/181] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:16.823030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8cad3-ce66-465e-910c-14afc46beba9 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:16.823498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8cad3-ce66-465e-910c-14afc46beba9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:16.823570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8cad3-ce66-465e-910c-14afc46beba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:16.845863 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d6b8cad3-ce66-465e-910c-14afc46beba9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:16.847099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d6b8cad3-ce66-465e-910c-14afc46beba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:16.929909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aefea-fe0d-41f5-afc8-d19de9a77e19 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:16.930291 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aefea-fe0d-41f5-afc8-d19de9a77e19 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:16.930503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aefea-fe0d-41f5-afc8-d19de9a77e19 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:16.931842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 90/182] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:16.937007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 93/183] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:16.938498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46799e-dd3e-4569-a79f-20be110cd039 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:16.938781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46799e-dd3e-4569-a79f-20be110cd039 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:16.939052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46799e-dd3e-4569-a79f-20be110cd039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:16.962240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-4e46799e-dd3e-4569-a79f-20be110cd039 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:16.963550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4e46799e-dd3e-4569-a79f-20be110cd039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:16.965701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8cad3-ce66-465e-910c-14afc46beba9 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:17.046582 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 91/184] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:17.055356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b057a-f1e8-402c-8e7f-4efd817f696b None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.055747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b057a-f1e8-402c-8e7f-4efd817f696b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.055950 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b057a-f1e8-402c-8e7f-4efd817f696b None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.057201 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 92/185] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:17.066197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44867f-2223-4419-9441-f191df3391e9 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.066197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44867f-2223-4419-9441-f191df3391e9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.066197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44867f-2223-4419-9441-f191df3391e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.069259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:17.086876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-cf44867f-2223-4419-9441-f191df3391e9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:17.088114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-cf44867f-2223-4419-9441-f191df3391e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:17.093628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8cad3-ce66-465e-910c-14afc46beba9 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:17.131458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c7cb9-dd9e-4070-ae49-727ba3517cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:17.132120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c7cb9-dd9e-4070-ae49-727ba3517cf7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.132408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c7cb9-dd9e-4070-ae49-727ba3517cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.132688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c7cb9-dd9e-4070-ae49-727ba3517cf7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.143803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c7cb9-dd9e-4070-ae49-727ba3517cf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-f89c7cb9-dd9e-4070-ae49-727ba3517cf7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:17.149270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:17.150261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:17.151093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 94/186] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:17.151782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c7cb9-dd9e-4070-ae49-727ba3517cf7 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:17.165478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 93/187] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:17.165478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44867f-2223-4419-9441-f191df3391e9 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:17.214240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:17.268516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c7cb9-dd9e-4070-ae49-727ba3517cf7 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:17.279823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44867f-2223-4419-9441-f191df3391e9 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:17.303655 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042f065-a473-4224-bb2d-e48e30069464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:17.304300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042f065-a473-4224-bb2d-e48e30069464 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.304583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042f065-a473-4224-bb2d-e48e30069464 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.304865 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042f065-a473-4224-bb2d-e48e30069464 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.307885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:17.320880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255f41d-26bd-472b-8765-c8b932a2215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:17.321034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042f065-a473-4224-bb2d-e48e30069464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:17.321111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f41d-26bd-472b-8765-c8b932a2215a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.321111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f41d-26bd-472b-8765-c8b932a2215a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.321111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f41d-26bd-472b-8765-c8b932a2215a None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.321111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255f41d-26bd-472b-8765-c8b932a2215a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:17.321111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255f41d-26bd-472b-8765-c8b932a2215a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:17.321111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255f41d-26bd-472b-8765-c8b932a2215a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-b042f065-a473-4224-bb2d-e48e30069464 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:17.322941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:17.323791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:17.324592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 95/188] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:21:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:17.325247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042f065-a473-4224-bb2d-e48e30069464 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:17.337309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 94/189] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:17.338752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255f41d-26bd-472b-8765-c8b932a2215a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:17.435107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042f065-a473-4224-bb2d-e48e30069464 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:17.446923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:17.450332 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255f41d-26bd-472b-8765-c8b932a2215a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:17.470245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb87ff7-3bc1-4e92-b6e7-6a32d43ec0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:17.470891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb87ff7-3bc1-4e92-b6e7-6a32d43ec0a3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.471175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb87ff7-3bc1-4e92-b6e7-6a32d43ec0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.471454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb87ff7-3bc1-4e92-b6e7-6a32d43ec0a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.472615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb87ff7-3bc1-4e92-b6e7-6a32d43ec0a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:17.474646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb87ff7-3bc1-4e92-b6e7-6a32d43ec0a3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:17.476227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 96/190] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:17.478855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb87ff7-3bc1-4e92-b6e7-6a32d43ec0a3 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:17.484863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010942e0-5931-4058-8163-ad8073619066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:17.485480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010942e0-5931-4058-8163-ad8073619066 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.485758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010942e0-5931-4058-8163-ad8073619066 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.486051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010942e0-5931-4058-8163-ad8073619066 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.486767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010942e0-5931-4058-8163-ad8073619066 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:17.487066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010942e0-5931-4058-8163-ad8073619066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:17.488426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010942e0-5931-4058-8163-ad8073619066 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:17.497312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 95/191] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:17.498556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436b07d-a37f-4833-9340-a3e3b939ca7f None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.498850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436b07d-a37f-4833-9340-a3e3b939ca7f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.499115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436b07d-a37f-4833-9340-a3e3b939ca7f None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.500309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 96/192] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:17.500980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436b07d-a37f-4833-9340-a3e3b939ca7f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:17.547409 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:17.584307 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb87ff7-3bc1-4e92-b6e7-6a32d43ec0a3 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:17.603155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436b07d-a37f-4833-9340-a3e3b939ca7f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:17.619490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ea273-3dfd-48b2-8141-3875cc4fe193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:17.620134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ea273-3dfd-48b2-8141-3875cc4fe193 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.620418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ea273-3dfd-48b2-8141-3875cc4fe193 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.620699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ea273-3dfd-48b2-8141-3875cc4fe193 None admin] PATH_INFO: `/v3/users/726553787fb341f485b8561971b880d1` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.634049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ea273-3dfd-48b2-8141-3875cc4fe193 None admin] RBAC: Authorizing `identity:get_user(user_id=726553787fb341f485b8561971b880d1)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:17.635519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ea273-3dfd-48b2-8141-3875cc4fe193 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:17.637156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 97/193] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:17 2022] GET /identity/v3/users/726553787fb341f485b8561971b880d1 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:17.637287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131cd195-ecc5-4bb6-be00-36b410341849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:17.637774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cd195-ecc5-4bb6-be00-36b410341849 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.637896 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ea273-3dfd-48b2-8141-3875cc4fe193 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:17.638086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cd195-ecc5-4bb6-be00-36b410341849 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.638368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cd195-ecc5-4bb6-be00-36b410341849 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.639050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131cd195-ecc5-4bb6-be00-36b410341849 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:17.640478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131cd195-ecc5-4bb6-be00-36b410341849 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:17.736983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ea273-3dfd-48b2-8141-3875cc4fe193 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:17.755841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 97/194] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:21:17 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:17.755841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994782c-3ac0-4e7c-900e-c508c33154f7 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.755841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994782c-3ac0-4e7c-900e-c508c33154f7 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.755841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994782c-3ac0-4e7c-900e-c508c33154f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.768531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f994782c-3ac0-4e7c-900e-c508c33154f7 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:17.769929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f994782c-3ac0-4e7c-900e-c508c33154f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:17.774461 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:17.787314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:17.787602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30e614-0ec9-45cf-8f84-8874c533634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:17.788246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30e614-0ec9-45cf-8f84-8874c533634f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.788534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30e614-0ec9-45cf-8f84-8874c533634f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.788809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30e614-0ec9-45cf-8f84-8874c533634f None admin] PATH_INFO: `/v3/users/726553787fb341f485b8561971b880d1` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.790256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30e614-0ec9-45cf-8f84-8874c533634f None admin] RBAC: Authorizing `identity:get_user(user_id=726553787fb341f485b8561971b880d1)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:17.791749 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30e614-0ec9-45cf-8f84-8874c533634f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:17.793649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 98/195] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:17 2022] GET /identity/v3/users/726553787fb341f485b8561971b880d1 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:17.794279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30e614-0ec9-45cf-8f84-8874c533634f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:17.856053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 98/196] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:17.857284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994782c-3ac0-4e7c-900e-c508c33154f7 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:17.915992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30e614-0ec9-45cf-8f84-8874c533634f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:17.955908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151fadc-a571-441f-a1e7-fcd7afdd29a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:17.956539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151fadc-a571-441f-a1e7-fcd7afdd29a2 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:17.956836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151fadc-a571-441f-a1e7-fcd7afdd29a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:17.957103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151fadc-a571-441f-a1e7-fcd7afdd29a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:17.985133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994782c-3ac0-4e7c-900e-c508c33154f7 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:18.029057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:18.030076 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b24731-491e-467d-8845-7ca4d758fe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:18.030740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b24731-491e-467d-8845-7ca4d758fe47 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:18.031023 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b24731-491e-467d-8845-7ca4d758fe47 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:18.031490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b24731-491e-467d-8845-7ca4d758fe47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:18.044338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151fadc-a571-441f-a1e7-fcd7afdd29a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:18.046032 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151fadc-a571-441f-a1e7-fcd7afdd29a2 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:18.058590 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:18.123914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b24731-491e-467d-8845-7ca4d758fe47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:18.126198 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b24731-491e-467d-8845-7ca4d758fe47 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:18.162819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 99/197] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:18.163677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151fadc-a571-441f-a1e7-fcd7afdd29a2 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:18.245614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 99/198] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:18.250892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b24731-491e-467d-8845-7ca4d758fe47 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:18.268527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:18.279676 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151fadc-a571-441f-a1e7-fcd7afdd29a2 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:18.303385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:18.318956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a61149-a335-4e23-9789-710b709bec49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:18.319609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a61149-a335-4e23-9789-710b709bec49 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:18.319948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a61149-a335-4e23-9789-710b709bec49 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:18.320225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a61149-a335-4e23-9789-710b709bec49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:18.331045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a61149-a335-4e23-9789-710b709bec49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-28a61149-a335-4e23-9789-710b709bec49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:18.336313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:18.337228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:18.338045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 100/199] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:18.338735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a61149-a335-4e23-9789-710b709bec49 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:18.361737 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b24731-491e-467d-8845-7ca4d758fe47 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:18.400334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7199f-b2c4-4fe7-ab6e-b799189360f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:18.400895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7199f-b2c4-4fe7-ab6e-b799189360f6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:18.401203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7199f-b2c4-4fe7-ab6e-b799189360f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:18.401476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7199f-b2c4-4fe7-ab6e-b799189360f6 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:18.404226 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7199f-b2c4-4fe7-ab6e-b799189360f6 None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:18.405663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7199f-b2c4-4fe7-ab6e-b799189360f6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:18.407502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 100/200] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:18 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:18.408264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7199f-b2c4-4fe7-ab6e-b799189360f6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:18.474673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a61149-a335-4e23-9789-710b709bec49 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:18.509957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61671c71-9b6b-4e86-ae5c-531b0d0e3300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:18.510623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61671c71-9b6b-4e86-ae5c-531b0d0e3300 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:18.510903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61671c71-9b6b-4e86-ae5c-531b0d0e3300 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:18.511174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61671c71-9b6b-4e86-ae5c-531b0d0e3300 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:18.512342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61671c71-9b6b-4e86-ae5c-531b0d0e3300 None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:18.513784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61671c71-9b6b-4e86-ae5c-531b0d0e3300 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:18.515438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 101/201] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:18 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:18.516191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61671c71-9b6b-4e86-ae5c-531b0d0e3300 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:18.528484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7199f-b2c4-4fe7-ab6e-b799189360f6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:18.537158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:18.546577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:18.571635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355d830-1f8a-4a5c-81a9-56f0518d19b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:18.572265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355d830-1f8a-4a5c-81a9-56f0518d19b9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:18.572634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355d830-1f8a-4a5c-81a9-56f0518d19b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:18.572909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355d830-1f8a-4a5c-81a9-56f0518d19b9 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:18.573626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355d830-1f8a-4a5c-81a9-56f0518d19b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:18.573930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355d830-1f8a-4a5c-81a9-56f0518d19b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:18.575354 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355d830-1f8a-4a5c-81a9-56f0518d19b9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:18.589691 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 101/202] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:18.590640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355d830-1f8a-4a5c-81a9-56f0518d19b9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:18.636558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61671c71-9b6b-4e86-ae5c-531b0d0e3300 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:18.671772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93299f-af6a-49d7-8417-5400b711ed50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:18.672330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93299f-af6a-49d7-8417-5400b711ed50 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:18.672621 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93299f-af6a-49d7-8417-5400b711ed50 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:18.672904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93299f-af6a-49d7-8417-5400b711ed50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:18.673705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93299f-af6a-49d7-8417-5400b711ed50 None admin] RBAC: Adding query filter params (role.id=677f038038444f229f95fc80c3166f24, scope.project.id=d64e4c325db8448dadb785b8df946d6c, user.id=726553787fb341f485b8561971b880d1) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:18.674038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93299f-af6a-49d7-8417-5400b711ed50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:18.675755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93299f-af6a-49d7-8417-5400b711ed50 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:18.682131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 102/203] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:18 2022] GET /identity/v3/role_assignments?user.id=726553787fb341f485b8561971b880d1&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=677f038038444f229f95fc80c3166f24 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:18.682971 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93299f-af6a-49d7-8417-5400b711ed50 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:18.712047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355d830-1f8a-4a5c-81a9-56f0518d19b9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:18.774410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:18.791408 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca04c3-5403-4500-b345-fcd89c5a8366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:18.792054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca04c3-5403-4500-b345-fcd89c5a8366 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:18.792333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca04c3-5403-4500-b345-fcd89c5a8366 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:18.792604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca04c3-5403-4500-b345-fcd89c5a8366 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:18.793398 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca04c3-5403-4500-b345-fcd89c5a8366 None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=d64e4c325db8448dadb785b8df946d6c, user.id=726553787fb341f485b8561971b880d1) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:18.793707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca04c3-5403-4500-b345-fcd89c5a8366 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:18.793931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93299f-af6a-49d7-8417-5400b711ed50 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:18.795187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca04c3-5403-4500-b345-fcd89c5a8366 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:18.809962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 102/204] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:18 2022] GET /identity/v3/role_assignments?user.id=726553787fb341f485b8561971b880d1&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:18.863713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f735b4-e8c0-4adf-b7af-33e04a261cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:18.864492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f735b4-e8c0-4adf-b7af-33e04a261cc2 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:18.864784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f735b4-e8c0-4adf-b7af-33e04a261cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:18.865068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f735b4-e8c0-4adf-b7af-33e04a261cc2 None admin] PATH_INFO: `/v3/users/36027a139e6e4266b13fc58e6db2c379` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:18.866519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f735b4-e8c0-4adf-b7af-33e04a261cc2 None admin] RBAC: Authorizing `identity:get_user(user_id=36027a139e6e4266b13fc58e6db2c379)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:18.867870 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f735b4-e8c0-4adf-b7af-33e04a261cc2 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:18.870116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 103/205] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:18 2022] GET /identity/v3/users/36027a139e6e4266b13fc58e6db2c379 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:18.875618 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca04c3-5403-4500-b345-fcd89c5a8366 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:18.937806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:19.003671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca04c3-5403-4500-b345-fcd89c5a8366 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:19.041380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01febc-3b21-43a9-ae1d-2db1e435b102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:19.042015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01febc-3b21-43a9-ae1d-2db1e435b102 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:19.042305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01febc-3b21-43a9-ae1d-2db1e435b102 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:19.042575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01febc-3b21-43a9-ae1d-2db1e435b102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:19.114406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01febc-3b21-43a9-ae1d-2db1e435b102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:19.115872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01febc-3b21-43a9-ae1d-2db1e435b102 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:19.178807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:19.219306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 103/206] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:19.225311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f735b4-e8c0-4adf-b7af-33e04a261cc2 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:19.343454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f735b4-e8c0-4adf-b7af-33e04a261cc2 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:19.378820 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a34d0f-1a7c-42fe-829b-452609e22807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:19.379528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a34d0f-1a7c-42fe-829b-452609e22807 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:19.379818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a34d0f-1a7c-42fe-829b-452609e22807 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:19.380099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a34d0f-1a7c-42fe-829b-452609e22807 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:19.381278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a34d0f-1a7c-42fe-829b-452609e22807 None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:19.382827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a34d0f-1a7c-42fe-829b-452609e22807 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:19.384707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 104/207] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:19 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:19.390384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01febc-3b21-43a9-ae1d-2db1e435b102 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:19.514502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01febc-3b21-43a9-ae1d-2db1e435b102 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:19.571041 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c697896-7202-4a79-809a-d4134d63a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:19.571674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c697896-7202-4a79-809a-d4134d63a522 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:19.571960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c697896-7202-4a79-809a-d4134d63a522 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:19.572236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c697896-7202-4a79-809a-d4134d63a522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:19.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c697896-7202-4a79-809a-d4134d63a522 None admin] RBAC: Adding query filter params (role.id=2ca1bca713f745ea9b40dc20cb5fa2a1, scope.project.id=6d126b37c8ae4111b880ecd5592237b8, user.id=36027a139e6e4266b13fc58e6db2c379) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:19.573404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c697896-7202-4a79-809a-d4134d63a522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:19.574882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c697896-7202-4a79-809a-d4134d63a522 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:19.587294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 104/208] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:19 2022] GET /identity/v3/role_assignments?user.id=36027a139e6e4266b13fc58e6db2c379&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=2ca1bca713f745ea9b40dc20cb5fa2a1 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:20.102376 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59052dbe-b06a-4284-a28f-829b3418db0c None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.102749 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59052dbe-b06a-4284-a28f-829b3418db0c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.102941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59052dbe-b06a-4284-a28f-829b3418db0c None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:20.104418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 105/209] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:20.109379 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee208-ea42-493b-a2b6-eaf7781f6813 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.109927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee208-ea42-493b-a2b6-eaf7781f6813 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.110234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee208-ea42-493b-a2b6-eaf7781f6813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:20.124392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:20.130288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-306ee208-ea42-493b-a2b6-eaf7781f6813 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:20.131474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-306ee208-ea42-493b-a2b6-eaf7781f6813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:20.206982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 105/210] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:20.225112 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703f15e-6d8f-4daf-adb1-ab41f62e1ccc None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.225398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703f15e-6d8f-4daf-adb1-ab41f62e1ccc None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.225904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703f15e-6d8f-4daf-adb1-ab41f62e1ccc None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:20.227289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 106/211] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:20.232086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a5645-b301-41e7-97ee-35e0e3da2c6c None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.232378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a5645-b301-41e7-97ee-35e0e3da2c6c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.232653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a5645-b301-41e7-97ee-35e0e3da2c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:20.236708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703f15e-6d8f-4daf-adb1-ab41f62e1ccc None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:20.258594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:20.264659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ae2a5645-b301-41e7-97ee-35e0e3da2c6c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:20.266556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ae2a5645-b301-41e7-97ee-35e0e3da2c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:20.343764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 106/212] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:20.361340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703f15e-6d8f-4daf-adb1-ab41f62e1ccc None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:20.368164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a5645-b301-41e7-97ee-35e0e3da2c6c None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:20.407952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994f73d-73f0-4750-854c-2ce617904b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:20.408637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994f73d-73f0-4750-854c-2ce617904b59 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.408923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994f73d-73f0-4750-854c-2ce617904b59 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.409197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994f73d-73f0-4750-854c-2ce617904b59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:20.423966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994f73d-73f0-4750-854c-2ce617904b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-7994f73d-73f0-4750-854c-2ce617904b59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:20.431405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:20.432281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:20.433064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 107/213] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:20.439906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994f73d-73f0-4750-854c-2ce617904b59 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:20.445785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:20.495402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a5645-b301-41e7-97ee-35e0e3da2c6c None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:20.529557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:20.550229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb03b7-3dc0-4d52-a531-172efec24605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:20.550892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb03b7-3dc0-4d52-a531-172efec24605 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.551806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb03b7-3dc0-4d52-a531-172efec24605 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.552188 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb03b7-3dc0-4d52-a531-172efec24605 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:20.554035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb03b7-3dc0-4d52-a531-172efec24605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:20.555496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb03b7-3dc0-4d52-a531-172efec24605 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:20.555857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994f73d-73f0-4750-854c-2ce617904b59 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:20.557338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 107/214] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:20.563472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb03b7-3dc0-4d52-a531-172efec24605 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:20.599953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b7311-dcaf-45c9-972c-11e1da4cfe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:20.600947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b7311-dcaf-45c9-972c-11e1da4cfe60 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.601608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b7311-dcaf-45c9-972c-11e1da4cfe60 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.602200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b7311-dcaf-45c9-972c-11e1da4cfe60 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:20.603705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b7311-dcaf-45c9-972c-11e1da4cfe60 None admin] RBAC: Adding query filter params (name=member) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:20.604154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b7311-dcaf-45c9-972c-11e1da4cfe60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:20.605985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b7311-dcaf-45c9-972c-11e1da4cfe60 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:20.618052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 108/215] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:20.623424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b7311-dcaf-45c9-972c-11e1da4cfe60 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:20.684592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb03b7-3dc0-4d52-a531-172efec24605 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:20.722546 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451d433-8704-46e0-af70-5b0650db93ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:20.724168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451d433-8704-46e0-af70-5b0650db93ca None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.724687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451d433-8704-46e0-af70-5b0650db93ca None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.725100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451d433-8704-46e0-af70-5b0650db93ca None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:20.726035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451d433-8704-46e0-af70-5b0650db93ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:20.727627 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451d433-8704-46e0-af70-5b0650db93ca None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:20.728640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b7311-dcaf-45c9-972c-11e1da4cfe60 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:20.736951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:20.764857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb0650-826c-49c1-8cc8-6d00f443e85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:20.765601 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb0650-826c-49c1-8cc8-6d00f443e85c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.765955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb0650-826c-49c1-8cc8-6d00f443e85c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.766328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb0650-826c-49c1-8cc8-6d00f443e85c None admin] PATH_INFO: `/v3/users/f5e09ac9486143b583ec1b1150ccaf48` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:20.779994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:20.783421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb0650-826c-49c1-8cc8-6d00f443e85c None admin] RBAC: Authorizing `identity:get_user(user_id=f5e09ac9486143b583ec1b1150ccaf48)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:20.784938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb0650-826c-49c1-8cc8-6d00f443e85c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:20.790190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 109/216] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:20 2022] GET /identity/v3/users/f5e09ac9486143b583ec1b1150ccaf48 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:20.792931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb0650-826c-49c1-8cc8-6d00f443e85c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:20.826260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 108/217] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:21:20 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:20.920161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb0650-826c-49c1-8cc8-6d00f443e85c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:20.960284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6409dde-7d2f-40c2-ab97-65262a35a334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:20.960979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6409dde-7d2f-40c2-ab97-65262a35a334 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.961264 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6409dde-7d2f-40c2-ab97-65262a35a334 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.961536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6409dde-7d2f-40c2-ab97-65262a35a334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:20.978839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6fb84-519b-4257-bde2-663db9d349bd None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.979153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6fb84-519b-4257-bde2-663db9d349bd None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.979410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6fb84-519b-4257-bde2-663db9d349bd None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:20.980932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 109/218] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:20.987353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa198f2b-2725-46a8-b77c-ccdacd4331e9 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:20.987665 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa198f2b-2725-46a8-b77c-ccdacd4331e9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:20.987994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa198f2b-2725-46a8-b77c-ccdacd4331e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:21.010085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-fa198f2b-2725-46a8-b77c-ccdacd4331e9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:21.011310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-fa198f2b-2725-46a8-b77c-ccdacd4331e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:21.044962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:21.048167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6409dde-7d2f-40c2-ab97-65262a35a334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:21.049666 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6409dde-7d2f-40c2-ab97-65262a35a334 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:21.105430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 110/219] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:21.126872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa198f2b-2725-46a8-b77c-ccdacd4331e9 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:21.174783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 110/220] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 21:21:21.180538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6409dde-7d2f-40c2-ab97-65262a35a334 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:21.269308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa198f2b-2725-46a8-b77c-ccdacd4331e9 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:21.295419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:21.295612 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:21.301694 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6409dde-7d2f-40c2-ab97-65262a35a334 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:21.317561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e9e0c-e68b-483a-8590-9171b35da707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:21.318462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e9e0c-e68b-483a-8590-9171b35da707 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:21.325936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e9e0c-e68b-483a-8590-9171b35da707 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:21.325936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e9e0c-e68b-483a-8590-9171b35da707 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:21.333424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e9e0c-e68b-483a-8590-9171b35da707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-349e9e0c-e68b-483a-8590-9171b35da707 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:21.340327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:21.341282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:21.342160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 111/221] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:21.349238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e9e0c-e68b-483a-8590-9171b35da707 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:21.349868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b088221-28e5-44e5-bcd9-5af8e7183490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:21.350700 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b088221-28e5-44e5-bcd9-5af8e7183490 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:21.350983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b088221-28e5-44e5-bcd9-5af8e7183490 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:21.351253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b088221-28e5-44e5-bcd9-5af8e7183490 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:21.354595 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b088221-28e5-44e5-bcd9-5af8e7183490 None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:21.356163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b088221-28e5-44e5-bcd9-5af8e7183490 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:21.365267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 111/222] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:21 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:21.365267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b088221-28e5-44e5-bcd9-5af8e7183490 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:21.456235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e9e0c-e68b-483a-8590-9171b35da707 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:21.493616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae397662-f76d-47ae-883a-e1f0738a51f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:21.494277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397662-f76d-47ae-883a-e1f0738a51f2 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:21.494560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397662-f76d-47ae-883a-e1f0738a51f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:21.494841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397662-f76d-47ae-883a-e1f0738a51f2 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:21.495823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae397662-f76d-47ae-883a-e1f0738a51f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:21.496137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae397662-f76d-47ae-883a-e1f0738a51f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:21.497626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae397662-f76d-47ae-883a-e1f0738a51f2 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:21.507727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 112/223] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:21.513249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae397662-f76d-47ae-883a-e1f0738a51f2 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:21.528300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b088221-28e5-44e5-bcd9-5af8e7183490 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:21.564667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:21.565244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cb9e9-35f1-47e1-97be-fcded57cdca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:21.566164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cb9e9-35f1-47e1-97be-fcded57cdca3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:21.567164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cb9e9-35f1-47e1-97be-fcded57cdca3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:21.567654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cb9e9-35f1-47e1-97be-fcded57cdca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:21.568826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cb9e9-35f1-47e1-97be-fcded57cdca3 None admin] RBAC: Adding query filter params (role.id=36291c067d1949cfae0d704585f036c8, scope.project.id=d64e4c325db8448dadb785b8df946d6c, user.id=f5e09ac9486143b583ec1b1150ccaf48) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:21.569369 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cb9e9-35f1-47e1-97be-fcded57cdca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:21.571074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cb9e9-35f1-47e1-97be-fcded57cdca3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:21.577009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 112/224] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:21 2022] GET /identity/v3/role_assignments?user.id=f5e09ac9486143b583ec1b1150ccaf48&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=36291c067d1949cfae0d704585f036c8 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:21.583761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20ccac-d4cb-4266-abe4-b1fa3cf6d6f9 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:21.584432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20ccac-d4cb-4266-abe4-b1fa3cf6d6f9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:21.584933 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20ccac-d4cb-4266-abe4-b1fa3cf6d6f9 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:21.590682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 113/225] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:21 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:21.606898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e3946-1a0a-4208-a442-604b9beab252 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:21.607527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e3946-1a0a-4208-a442-604b9beab252 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:21.608003 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e3946-1a0a-4208-a442-604b9beab252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:21.636370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae397662-f76d-47ae-883a-e1f0738a51f2 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:21.646038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-403e3946-1a0a-4208-a442-604b9beab252 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:21.647900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-403e3946-1a0a-4208-a442-604b9beab252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:21.678380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f797633-82ca-447c-9bfb-95f09bb736a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:21.679051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f797633-82ca-447c-9bfb-95f09bb736a1 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:21.680060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f797633-82ca-447c-9bfb-95f09bb736a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:21.680401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f797633-82ca-447c-9bfb-95f09bb736a1 None admin] PATH_INFO: `/v3/users/726553787fb341f485b8561971b880d1` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:21.682142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f797633-82ca-447c-9bfb-95f09bb736a1 None admin] RBAC: Authorizing `identity:get_user(user_id=726553787fb341f485b8561971b880d1)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:21.683966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:21.685205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f797633-82ca-447c-9bfb-95f09bb736a1 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:21.687729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 113/226] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:21 2022] GET /identity/v3/users/726553787fb341f485b8561971b880d1 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:21.699094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f797633-82ca-447c-9bfb-95f09bb736a1 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:21.745830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 114/227] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:21.771905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e3946-1a0a-4208-a442-604b9beab252 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:21.890141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e3946-1a0a-4208-a442-604b9beab252 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:21.892422 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f797633-82ca-447c-9bfb-95f09bb736a1 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:21.918381 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:21.932266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc086d-eaeb-4487-82ee-10437ad350e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:21.932924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc086d-eaeb-4487-82ee-10437ad350e4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:21.933206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc086d-eaeb-4487-82ee-10437ad350e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:21.933483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc086d-eaeb-4487-82ee-10437ad350e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:21.946471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc086d-eaeb-4487-82ee-10437ad350e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-dfdc086d-eaeb-4487-82ee-10437ad350e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:21.952058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:21.952954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:21.953759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 115/228] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:21:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:21.959558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc086d-eaeb-4487-82ee-10437ad350e4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:21.983418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a72bd8-97e8-4d1a-863e-c268e208fa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:21.984113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72bd8-97e8-4d1a-863e-c268e208fa87 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:21.984363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72bd8-97e8-4d1a-863e-c268e208fa87 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:21.984638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72bd8-97e8-4d1a-863e-c268e208fa87 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:21.986880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a72bd8-97e8-4d1a-863e-c268e208fa87 None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:21.990824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a72bd8-97e8-4d1a-863e-c268e208fa87 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:21.999707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 114/229] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:21 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:22.000489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a72bd8-97e8-4d1a-863e-c268e208fa87 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:22.014396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:22.088544 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc086d-eaeb-4487-82ee-10437ad350e4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:22.130117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22999595-bb17-400b-aa85-44766af1f0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:22.130834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22999595-bb17-400b-aa85-44766af1f0ff None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:22.131186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22999595-bb17-400b-aa85-44766af1f0ff None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:22.131531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22999595-bb17-400b-aa85-44766af1f0ff None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:22.132315 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22999595-bb17-400b-aa85-44766af1f0ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:22.132689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22999595-bb17-400b-aa85-44766af1f0ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:22.134177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22999595-bb17-400b-aa85-44766af1f0ff None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:22.142800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 116/230] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:22.148773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22999595-bb17-400b-aa85-44766af1f0ff None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:22.185114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a72bd8-97e8-4d1a-863e-c268e208fa87 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:22.232607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df19407-62ff-4201-8d76-d65f040b2d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:22.233298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df19407-62ff-4201-8d76-d65f040b2d50 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:22.233582 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df19407-62ff-4201-8d76-d65f040b2d50 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:22.233860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df19407-62ff-4201-8d76-d65f040b2d50 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c/users/726553787fb341f485b8561971b880d1/roles/677f038038444f229f95fc80c3166f24` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:22.240129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:22.240269 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df19407-62ff-4201-8d76-d65f040b2d50 None admin] RBAC: Authorizing `identity:create_grant(project_id=d64e4c325db8448dadb785b8df946d6c, user_id=726553787fb341f485b8561971b880d1, role_id=677f038038444f229f95fc80c3166f24)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:22.241690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df19407-62ff-4201-8d76-d65f040b2d50 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:22.263911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22999595-bb17-400b-aa85-44766af1f0ff None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:22.297914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a987b-006c-4227-a442-b1d28caca44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:22.298701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a987b-006c-4227-a442-b1d28caca44c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:22.298887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a987b-006c-4227-a442-b1d28caca44c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:22.299194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a987b-006c-4227-a442-b1d28caca44c None admin] PATH_INFO: `/v3/users/726553787fb341f485b8561971b880d1` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:22.300483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a987b-006c-4227-a442-b1d28caca44c None admin] RBAC: Authorizing `identity:get_user(user_id=726553787fb341f485b8561971b880d1)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:22.302163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a987b-006c-4227-a442-b1d28caca44c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:22.304082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 117/231] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:22 2022] GET /identity/v3/users/726553787fb341f485b8561971b880d1 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:22.309857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a987b-006c-4227-a442-b1d28caca44c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:22.323614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 115/232] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:21:21 2022] PUT /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c/users/726553787fb341f485b8561971b880d1/roles/677f038038444f229f95fc80c3166f24 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:22.452861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a987b-006c-4227-a442-b1d28caca44c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:22.511586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c352fb-abae-4e5c-8bb6-ea006cdc2411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:22.512244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c352fb-abae-4e5c-8bb6-ea006cdc2411 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:22.512529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c352fb-abae-4e5c-8bb6-ea006cdc2411 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:22.512806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c352fb-abae-4e5c-8bb6-ea006cdc2411 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:22.515920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c352fb-abae-4e5c-8bb6-ea006cdc2411 None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:22.517331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c352fb-abae-4e5c-8bb6-ea006cdc2411 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:22.519374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 118/233] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:22 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:22.525556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df19407-62ff-4201-8d76-d65f040b2d50 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:22.577308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:22.654777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df19407-62ff-4201-8d76-d65f040b2d50 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:22.706403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad75c7-e3aa-4155-82d4-28d0608c0744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:22.707550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad75c7-e3aa-4155-82d4-28d0608c0744 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:22.708056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad75c7-e3aa-4155-82d4-28d0608c0744 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:22.708527 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad75c7-e3aa-4155-82d4-28d0608c0744 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c/users/726553787fb341f485b8561971b880d1/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:22.712513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad75c7-e3aa-4155-82d4-28d0608c0744 None admin] RBAC: Authorizing `identity:create_grant(project_id=d64e4c325db8448dadb785b8df946d6c, user_id=726553787fb341f485b8561971b880d1, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:22.714203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad75c7-e3aa-4155-82d4-28d0608c0744 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:22.847407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 116/234] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:21:22 2022] PUT /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c/users/726553787fb341f485b8561971b880d1/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:23.452647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386768-fedb-429d-9757-d1841aae7a86 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:23.453157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386768-fedb-429d-9757-d1841aae7a86 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:23.453230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386768-fedb-429d-9757-d1841aae7a86 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:23.454879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 119/235] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:23.460314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec4735-ffa3-4aee-9e45-5933f94230ed None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:23.460850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec4735-ffa3-4aee-9e45-5933f94230ed None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:23.461322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec4735-ffa3-4aee-9e45-5933f94230ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:23.491138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-67ec4735-ffa3-4aee-9e45-5933f94230ed None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:23.492680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-67ec4735-ffa3-4aee-9e45-5933f94230ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:23.503786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e25203-f80c-40bf-a7cc-1057d0313375 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:23.504644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e25203-f80c-40bf-a7cc-1057d0313375 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:23.505075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e25203-f80c-40bf-a7cc-1057d0313375 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:23.507018 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 120/236] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:23.522770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b429f-f67c-4986-adcb-d83f6d9ebd5d None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:23.524046 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b429f-f67c-4986-adcb-d83f6d9ebd5d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:23.524527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b429f-f67c-4986-adcb-d83f6d9ebd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:23.555629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-e37b429f-f67c-4986-adcb-d83f6d9ebd5d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:23.557155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e37b429f-f67c-4986-adcb-d83f6d9ebd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:23.578824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 117/237] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:23.588044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:23.606432 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec4735-ffa3-4aee-9e45-5933f94230ed None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:23.645265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:23.700385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 121/238] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:23.730397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b429f-f67c-4986-adcb-d83f6d9ebd5d None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:23.756533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec4735-ffa3-4aee-9e45-5933f94230ed None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:23.823238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f3d29-2403-46b3-92ea-1e79f897c16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:23.824800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f3d29-2403-46b3-92ea-1e79f897c16e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:23.825495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f3d29-2403-46b3-92ea-1e79f897c16e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:23.826323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f3d29-2403-46b3-92ea-1e79f897c16e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:23.829330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f3d29-2403-46b3-92ea-1e79f897c16e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:23.831025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f3d29-2403-46b3-92ea-1e79f897c16e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:23.835620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 118/239] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:23.842782 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f3d29-2403-46b3-92ea-1e79f897c16e None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:23.907147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b429f-f67c-4986-adcb-d83f6d9ebd5d None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:23.966110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5437ee-c7a8-414d-a34e-78205e52beda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:23.967147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5437ee-c7a8-414d-a34e-78205e52beda None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:23.967664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5437ee-c7a8-414d-a34e-78205e52beda None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:23.968133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5437ee-c7a8-414d-a34e-78205e52beda None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:23.981048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5437ee-c7a8-414d-a34e-78205e52beda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-df5437ee-c7a8-414d-a34e-78205e52beda None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:23.986547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:23.987477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:23.988287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:23.991061 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 122/240] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:24.004580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5437ee-c7a8-414d-a34e-78205e52beda None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:24.028927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:24.036175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:24.042640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f3d29-2403-46b3-92ea-1e79f897c16e None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:24.085112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec639b6d-91f6-4f67-b6b0-8e0d1936f3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:24.085766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec639b6d-91f6-4f67-b6b0-8e0d1936f3bb None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.086083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec639b6d-91f6-4f67-b6b0-8e0d1936f3bb None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.086363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec639b6d-91f6-4f67-b6b0-8e0d1936f3bb None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.087302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec639b6d-91f6-4f67-b6b0-8e0d1936f3bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:24.088611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec639b6d-91f6-4f67-b6b0-8e0d1936f3bb None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:24.166739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5437ee-c7a8-414d-a34e-78205e52beda None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:24.181974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 119/241] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:21:23 2022] POST /identity/v3/users => generated 293 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:21:24.227089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35473d61-f4e9-4a6c-9cc9-b3d8e6a4d564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:24.227737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35473d61-f4e9-4a6c-9cc9-b3d8e6a4d564 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.228019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35473d61-f4e9-4a6c-9cc9-b3d8e6a4d564 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.228296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35473d61-f4e9-4a6c-9cc9-b3d8e6a4d564 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.229025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35473d61-f4e9-4a6c-9cc9-b3d8e6a4d564 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:24.229332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35473d61-f4e9-4a6c-9cc9-b3d8e6a4d564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:24.230908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35473d61-f4e9-4a6c-9cc9-b3d8e6a4d564 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:24.247873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 123/242] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:24.253619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec639b6d-91f6-4f67-b6b0-8e0d1936f3bb None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:24.388390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec639b6d-91f6-4f67-b6b0-8e0d1936f3bb None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:24.432095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615bf06-8085-4f00-9c77-f2f51c845c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:24.433137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615bf06-8085-4f00-9c77-f2f51c845c79 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.433647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615bf06-8085-4f00-9c77-f2f51c845c79 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.434153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615bf06-8085-4f00-9c77-f2f51c845c79 None admin] PATH_INFO: `/v3/users/8246203d30344a509f48280c9f8ff7a3` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.439593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f7ccd-a81c-4b9c-84ac-8d491b9b940e None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.439847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f7ccd-a81c-4b9c-84ac-8d491b9b940e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.440128 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f7ccd-a81c-4b9c-84ac-8d491b9b940e None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.441471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 124/243] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:24.450763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523641b-622d-40d4-875c-4701a10adf00 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.451052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523641b-622d-40d4-875c-4701a10adf00 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.451332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523641b-622d-40d4-875c-4701a10adf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.454738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615bf06-8085-4f00-9c77-f2f51c845c79 None admin] RBAC: Authorizing `identity:get_user(user_id=8246203d30344a509f48280c9f8ff7a3)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:24.456416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615bf06-8085-4f00-9c77-f2f51c845c79 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:24.459192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 120/244] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:24 2022] GET /identity/v3/users/8246203d30344a509f48280c9f8ff7a3 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:24.466420 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615bf06-8085-4f00-9c77-f2f51c845c79 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:24.476096 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-8523641b-622d-40d4-875c-4701a10adf00 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:24.477496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8523641b-622d-40d4-875c-4701a10adf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:24.481085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:24.559336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:24.564649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 125/245] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:24.589996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523641b-622d-40d4-875c-4701a10adf00 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:24.627429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615bf06-8085-4f00-9c77-f2f51c845c79 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:24.681513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86548219-e322-42d9-8b51-0b016686455d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:24.682637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86548219-e322-42d9-8b51-0b016686455d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.683128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86548219-e322-42d9-8b51-0b016686455d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.683637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86548219-e322-42d9-8b51-0b016686455d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.740598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523641b-622d-40d4-875c-4701a10adf00 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:24.765757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86548219-e322-42d9-8b51-0b016686455d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:24.767285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86548219-e322-42d9-8b51-0b016686455d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:24.780676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:24.792083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc65a8-8f24-42bc-bc7d-f2f43d4cef09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:24.793082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc65a8-8f24-42bc-bc7d-f2f43d4cef09 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.794047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc65a8-8f24-42bc-bc7d-f2f43d4cef09 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.794560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc65a8-8f24-42bc-bc7d-f2f43d4cef09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.820111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc65a8-8f24-42bc-bc7d-f2f43d4cef09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-61fc65a8-8f24-42bc-bc7d-f2f43d4cef09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:24.825657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:24.826789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:24.827590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:24.829886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 126/246] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:24.831557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb23bf-735d-4ac7-81d3-4c98256b61a2 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.832673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb23bf-735d-4ac7-81d3-4c98256b61a2 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.833300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb23bf-735d-4ac7-81d3-4c98256b61a2 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.835133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 127/247] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:24.836059 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb23bf-735d-4ac7-81d3-4c98256b61a2 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:24.947453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 121/248] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:24.951312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5837418-2946-4278-9154-bc24f68580ac None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:24.955003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5837418-2946-4278-9154-bc24f68580ac None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:24.955524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5837418-2946-4278-9154-bc24f68580ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:24.970844 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:24.986298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-d5837418-2946-4278-9154-bc24f68580ac None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:24.990918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d5837418-2946-4278-9154-bc24f68580ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:25.047979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb23bf-735d-4ac7-81d3-4c98256b61a2 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:25.109457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 122/249] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:25.110653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5837418-2946-4278-9154-bc24f68580ac None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:25.122658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1f4ef-8221-450a-aede-b34c53213927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:25.123333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1f4ef-8221-450a-aede-b34c53213927 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.123614 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1f4ef-8221-450a-aede-b34c53213927 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.123888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1f4ef-8221-450a-aede-b34c53213927 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.126528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1f4ef-8221-450a-aede-b34c53213927 None admin] RBAC: Adding query filter params (name=member) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:25.126847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1f4ef-8221-450a-aede-b34c53213927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:25.128251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1f4ef-8221-450a-aede-b34c53213927 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:25.147671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 128/250] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:25.149911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1f4ef-8221-450a-aede-b34c53213927 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:25.190463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:25.306983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5837418-2946-4278-9154-bc24f68580ac None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:25.352514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1f4ef-8221-450a-aede-b34c53213927 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:25.375592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c67d71-a627-450f-bcc4-6b0906981dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:25.376241 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c67d71-a627-450f-bcc4-6b0906981dfc None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.376523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c67d71-a627-450f-bcc4-6b0906981dfc None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.376796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c67d71-a627-450f-bcc4-6b0906981dfc None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.378771 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c67d71-a627-450f-bcc4-6b0906981dfc None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:25.380816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c67d71-a627-450f-bcc4-6b0906981dfc None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:25.392840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 123/251] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:25 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:25.393702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c67d71-a627-450f-bcc4-6b0906981dfc None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:25.407706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:25.430561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986431a-a71a-4f57-a226-034e3624faef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:25.431209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986431a-a71a-4f57-a226-034e3624faef None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.431492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986431a-a71a-4f57-a226-034e3624faef None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.431764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986431a-a71a-4f57-a226-034e3624faef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.455274 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986431a-a71a-4f57-a226-034e3624faef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-e986431a-a71a-4f57-a226-034e3624faef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:25.463830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:25.464736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:25.465551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:25.469462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 129/252] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:25.470604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986431a-a71a-4f57-a226-034e3624faef None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:25.572627 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c67d71-a627-450f-bcc4-6b0906981dfc None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:25.583735 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:25.613604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c9d24-e694-4095-a97c-7250e514247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:25.614092 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986431a-a71a-4f57-a226-034e3624faef None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:25.614761 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c9d24-e694-4095-a97c-7250e514247d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.615322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c9d24-e694-4095-a97c-7250e514247d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.615815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c9d24-e694-4095-a97c-7250e514247d None admin] PATH_INFO: `/v3/users/f5e09ac9486143b583ec1b1150ccaf48` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.617667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c9d24-e694-4095-a97c-7250e514247d None admin] RBAC: Authorizing `identity:get_user(user_id=f5e09ac9486143b583ec1b1150ccaf48)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:25.620141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c9d24-e694-4095-a97c-7250e514247d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:25.622214 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 124/253] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:25 2022] GET /identity/v3/users/f5e09ac9486143b583ec1b1150ccaf48 => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:25.623126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c9d24-e694-4095-a97c-7250e514247d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:25.654978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ad306-da24-4269-8ced-8fbc34c94f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:25.656536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ad306-da24-4269-8ced-8fbc34c94f68 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.656970 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ad306-da24-4269-8ced-8fbc34c94f68 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.657389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ad306-da24-4269-8ced-8fbc34c94f68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.659021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ad306-da24-4269-8ced-8fbc34c94f68 None admin] RBAC: Adding query filter params (role.id=2ca1bca713f745ea9b40dc20cb5fa2a1, scope.project.id=d64e4c325db8448dadb785b8df946d6c, user.id=8246203d30344a509f48280c9f8ff7a3) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:25.659512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ad306-da24-4269-8ced-8fbc34c94f68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:25.662973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ad306-da24-4269-8ced-8fbc34c94f68 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:25.670723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 130/254] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:25 2022] GET /identity/v3/role_assignments?user.id=8246203d30344a509f48280c9f8ff7a3&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=2ca1bca713f745ea9b40dc20cb5fa2a1 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:25.672156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f405f-12f8-46f0-9811-4e41ba693705 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.672575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f405f-12f8-46f0-9811-4e41ba693705 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.673009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f405f-12f8-46f0-9811-4e41ba693705 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.674426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 131/255] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:25.675280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f405f-12f8-46f0-9811-4e41ba693705 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:25.756181 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:25.813530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f405f-12f8-46f0-9811-4e41ba693705 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:25.854939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84199daa-81fd-4fc8-851e-0bd79267300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:25.856333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84199daa-81fd-4fc8-851e-0bd79267300f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.856803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84199daa-81fd-4fc8-851e-0bd79267300f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.857380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84199daa-81fd-4fc8-851e-0bd79267300f None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.858992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84199daa-81fd-4fc8-851e-0bd79267300f None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:25.861245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84199daa-81fd-4fc8-851e-0bd79267300f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:25.864718 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 132/256] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:25 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:25.864939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c9d24-e694-4095-a97c-7250e514247d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:25.867110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac39a4e-0467-4b7a-a7d9-633236660159 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.867606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac39a4e-0467-4b7a-a7d9-633236660159 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.869090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac39a4e-0467-4b7a-a7d9-633236660159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.893107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-7ac39a4e-0467-4b7a-a7d9-633236660159 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:25.896421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-7ac39a4e-0467-4b7a-a7d9-633236660159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:25.904019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fa822-2ef9-448f-a1a5-fb8caefd1e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:25.904760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fa822-2ef9-448f-a1a5-fb8caefd1e2b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:25.905105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fa822-2ef9-448f-a1a5-fb8caefd1e2b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:25.905448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fa822-2ef9-448f-a1a5-fb8caefd1e2b None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:25.906332 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fa822-2ef9-448f-a1a5-fb8caefd1e2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:25.907216 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fa822-2ef9-448f-a1a5-fb8caefd1e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:25.909116 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fa822-2ef9-448f-a1a5-fb8caefd1e2b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:25.920275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 125/257] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:25.921044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fa822-2ef9-448f-a1a5-fb8caefd1e2b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:25.998370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:26.041507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 133/258] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:26.046859 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac39a4e-0467-4b7a-a7d9-633236660159 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:26.131484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fa822-2ef9-448f-a1a5-fb8caefd1e2b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:26.163393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:26.191251 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85979432-56a6-448d-b047-760762bc7a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:26.192642 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85979432-56a6-448d-b047-760762bc7a12 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:26.192929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85979432-56a6-448d-b047-760762bc7a12 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:26.193186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85979432-56a6-448d-b047-760762bc7a12 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c/users/f5e09ac9486143b583ec1b1150ccaf48/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:26.196355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85979432-56a6-448d-b047-760762bc7a12 None admin] RBAC: Authorizing `identity:create_grant(project_id=d64e4c325db8448dadb785b8df946d6c, user_id=f5e09ac9486143b583ec1b1150ccaf48, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:26.197783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85979432-56a6-448d-b047-760762bc7a12 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:26.243060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac39a4e-0467-4b7a-a7d9-633236660159 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:26.279746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f70ea8-ab06-4100-9bc8-0104b236b1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:26.280404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70ea8-ab06-4100-9bc8-0104b236b1f4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:26.280683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70ea8-ab06-4100-9bc8-0104b236b1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:26.280954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70ea8-ab06-4100-9bc8-0104b236b1f4 None admin] PATH_INFO: `/v3/users/726553787fb341f485b8561971b880d1` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:26.286837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f70ea8-ab06-4100-9bc8-0104b236b1f4 None admin] RBAC: Authorizing `identity:get_user(user_id=726553787fb341f485b8561971b880d1)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:26.290373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 126/259] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:21:25 2022] PUT /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c/users/f5e09ac9486143b583ec1b1150ccaf48/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:26.291273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85979432-56a6-448d-b047-760762bc7a12 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:26.296635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f70ea8-ab06-4100-9bc8-0104b236b1f4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:26.298897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 134/260] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:26 2022] GET /identity/v3/users/726553787fb341f485b8561971b880d1 => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:26.305623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f70ea8-ab06-4100-9bc8-0104b236b1f4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:26.514368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:26.534191 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85979432-56a6-448d-b047-760762bc7a12 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:26.547076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f70ea8-ab06-4100-9bc8-0104b236b1f4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:26.572450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:26.607456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13cc0d-e923-492e-9464-952621abcdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:26.608119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13cc0d-e923-492e-9464-952621abcdc8 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:26.608389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13cc0d-e923-492e-9464-952621abcdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:26.608669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13cc0d-e923-492e-9464-952621abcdc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:26.613267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a38c6-b68a-460f-8e74-851d6a42e4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:26.614128 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a38c6-b68a-460f-8e74-851d6a42e4a0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:26.614656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a38c6-b68a-460f-8e74-851d6a42e4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:26.614656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a38c6-b68a-460f-8e74-851d6a42e4a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:26.638220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13cc0d-e923-492e-9464-952621abcdc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-2d13cc0d-e923-492e-9464-952621abcdc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:26.643181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:26.644064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:26.645153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 127/261] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:21:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:26.651372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13cc0d-e923-492e-9464-952621abcdc8 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:26.767235 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a38c6-b68a-460f-8e74-851d6a42e4a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:26.768660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a38c6-b68a-460f-8e74-851d6a42e4a0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:26.877902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13cc0d-e923-492e-9464-952621abcdc8 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:26.938367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840245e-bcbb-4fce-9297-14fa2cb63120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:26.938998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840245e-bcbb-4fce-9297-14fa2cb63120 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:26.939655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840245e-bcbb-4fce-9297-14fa2cb63120 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:26.940484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840245e-bcbb-4fce-9297-14fa2cb63120 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:26.943029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840245e-bcbb-4fce-9297-14fa2cb63120 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:26.943430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840245e-bcbb-4fce-9297-14fa2cb63120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:26.945664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840245e-bcbb-4fce-9297-14fa2cb63120 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:26.965718 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 135/262] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 21:21:26.968068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52185ffe-aecf-4083-a5ed-e0ba2f96befa None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:26.968369 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52185ffe-aecf-4083-a5ed-e0ba2f96befa None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:26.968651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52185ffe-aecf-4083-a5ed-e0ba2f96befa None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:26.969927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 136/263] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:26.975137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 128/264] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:26.976152 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52185ffe-aecf-4083-a5ed-e0ba2f96befa None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:26.984231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01462c52-88b0-47f6-a3f2-1d5c4f27fb7e None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:26.984564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01462c52-88b0-47f6-a3f2-1d5c4f27fb7e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:26.985039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01462c52-88b0-47f6-a3f2-1d5c4f27fb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:27.018895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:27.027033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-01462c52-88b0-47f6-a3f2-1d5c4f27fb7e None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:27.028258 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-01462c52-88b0-47f6-a3f2-1d5c4f27fb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:27.070696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:27.121757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52185ffe-aecf-4083-a5ed-e0ba2f96befa None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:27.136888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 129/265] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 04 21:21:27.138100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01462c52-88b0-47f6-a3f2-1d5c4f27fb7e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:27.163692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552e7bf-1ecf-4b93-a9ef-856d6b17634a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:27.164544 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552e7bf-1ecf-4b93-a9ef-856d6b17634a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:27.164851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552e7bf-1ecf-4b93-a9ef-856d6b17634a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:27.165182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552e7bf-1ecf-4b93-a9ef-856d6b17634a None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:27.166821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552e7bf-1ecf-4b93-a9ef-856d6b17634a None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:27.168545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552e7bf-1ecf-4b93-a9ef-856d6b17634a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:27.170779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 137/266] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:26 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:27.172531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef0821-1761-4310-993f-31aa2fa00a6c None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:27.172806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef0821-1761-4310-993f-31aa2fa00a6c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:27.173242 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef0821-1761-4310-993f-31aa2fa00a6c None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:27.174743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 138/267] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:27.175227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef0821-1761-4310-993f-31aa2fa00a6c None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:27.254731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01462c52-88b0-47f6-a3f2-1d5c4f27fb7e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:27.292680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:27.307505 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68f4bc-b234-4583-941e-550581c93fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:27.308223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68f4bc-b234-4583-941e-550581c93fab None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:27.308508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68f4bc-b234-4583-941e-550581c93fab None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:27.308781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68f4bc-b234-4583-941e-550581c93fab None admin] PATH_INFO: `/v3/users/726553787fb341f485b8561971b880d1` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:27.315299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68f4bc-b234-4583-941e-550581c93fab None admin] RBAC: Authorizing `identity:get_user(user_id=726553787fb341f485b8561971b880d1)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:27.317751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68f4bc-b234-4583-941e-550581c93fab None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:27.318676 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef0821-1761-4310-993f-31aa2fa00a6c None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:27.320020 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 130/268] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:27 2022] GET /identity/v3/users/726553787fb341f485b8561971b880d1 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:27.325061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68f4bc-b234-4583-941e-550581c93fab None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:27.354604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:27.370106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ee5fb-c1a0-424c-bd75-c0996d9fb7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:27.370857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ee5fb-c1a0-424c-bd75-c0996d9fb7c7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:27.371259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ee5fb-c1a0-424c-bd75-c0996d9fb7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:27.371603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ee5fb-c1a0-424c-bd75-c0996d9fb7c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:27.383897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ee5fb-c1a0-424c-bd75-c0996d9fb7c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-f38ee5fb-c1a0-424c-bd75-c0996d9fb7c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:27.391815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:27.392708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:27.393509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:27.398436 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 139/269] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:27.399773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921e323-4a9f-4517-a161-35b5d0c40555 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:27.400126 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921e323-4a9f-4517-a161-35b5d0c40555 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:27.400472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921e323-4a9f-4517-a161-35b5d0c40555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:27.419774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-2921e323-4a9f-4517-a161-35b5d0c40555 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:27.421125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2921e323-4a9f-4517-a161-35b5d0c40555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:27.460889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68f4bc-b234-4583-941e-550581c93fab None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:27.499909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5ada9-edbc-401c-984a-23e36ea1b7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:27.500671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5ada9-edbc-401c-984a-23e36ea1b7b0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:27.501016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5ada9-edbc-401c-984a-23e36ea1b7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:27.501363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5ada9-edbc-401c-984a-23e36ea1b7b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:27.502333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5ada9-edbc-401c-984a-23e36ea1b7b0 None admin] RBAC: Adding query filter params (role.id=677f038038444f229f95fc80c3166f24, scope.project.id=d64e4c325db8448dadb785b8df946d6c, user.id=726553787fb341f485b8561971b880d1) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:27.502711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5ada9-edbc-401c-984a-23e36ea1b7b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:27.504371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5ada9-edbc-401c-984a-23e36ea1b7b0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:27.515135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 140/270] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:27.516271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921e323-4a9f-4517-a161-35b5d0c40555 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:27.518554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 131/271] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:27 2022] GET /identity/v3/role_assignments?user.id=726553787fb341f485b8561971b880d1&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=677f038038444f229f95fc80c3166f24 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:27.519711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5ada9-edbc-401c-984a-23e36ea1b7b0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:27.622148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:27.667713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921e323-4a9f-4517-a161-35b5d0c40555 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:27.717640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de5b44-5983-45ed-b6a4-d1f1b063f00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:27.718631 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de5b44-5983-45ed-b6a4-d1f1b063f00e None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:27.719125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de5b44-5983-45ed-b6a4-d1f1b063f00e None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:27.720298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de5b44-5983-45ed-b6a4-d1f1b063f00e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:27.747805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:27.786704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5ada9-edbc-401c-984a-23e36ea1b7b0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:27.819968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de5b44-5983-45ed-b6a4-d1f1b063f00e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:27.822557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de5b44-5983-45ed-b6a4-d1f1b063f00e None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:27.832569 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca641f-f656-4694-9162-682d4e470437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:27.833305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca641f-f656-4694-9162-682d4e470437 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:27.833706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca641f-f656-4694-9162-682d4e470437 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:27.834099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca641f-f656-4694-9162-682d4e470437 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:27.834938 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca641f-f656-4694-9162-682d4e470437 None admin] RBAC: Adding query filter params (name=member) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:27.835321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca641f-f656-4694-9162-682d4e470437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:27.837853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca641f-f656-4694-9162-682d4e470437 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:27.849444 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 132/272] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:27.854443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca641f-f656-4694-9162-682d4e470437 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:27.916255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:27.955197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 141/273] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:27.960959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de5b44-5983-45ed-b6a4-d1f1b063f00e None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:27.983887 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca641f-f656-4694-9162-682d4e470437 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:28.029155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43faf53d-fb09-448a-8eb1-8571fbf0f51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:28.029838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43faf53d-fb09-448a-8eb1-8571fbf0f51b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:28.030153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43faf53d-fb09-448a-8eb1-8571fbf0f51b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:28.030429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43faf53d-fb09-448a-8eb1-8571fbf0f51b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:28.031601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43faf53d-fb09-448a-8eb1-8571fbf0f51b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:28.033775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43faf53d-fb09-448a-8eb1-8571fbf0f51b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:28.036350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 133/274] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:28.037281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43faf53d-fb09-448a-8eb1-8571fbf0f51b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:28.057794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:28.094384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de5b44-5983-45ed-b6a4-d1f1b063f00e None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:28.138353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063fc2e-0032-493a-9408-bf6c60e74a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:28.139036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063fc2e-0032-493a-9408-bf6c60e74a08 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:28.139320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063fc2e-0032-493a-9408-bf6c60e74a08 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:28.139596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063fc2e-0032-493a-9408-bf6c60e74a08 None admin] PATH_INFO: `/v3/users/2f393ec159a4473eb4e0555cad6afaa2` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:28.157610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063fc2e-0032-493a-9408-bf6c60e74a08 None admin] RBAC: Authorizing `identity:get_user(user_id=2f393ec159a4473eb4e0555cad6afaa2)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:28.159599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063fc2e-0032-493a-9408-bf6c60e74a08 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:28.162031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 142/275] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:27 2022] GET /identity/v3/users/2f393ec159a4473eb4e0555cad6afaa2 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:28.166136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063fc2e-0032-493a-9408-bf6c60e74a08 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:28.173098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43faf53d-fb09-448a-8eb1-8571fbf0f51b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:28.204890 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:28.209778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9030933-1741-459f-8ba6-2fbca05eab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:28.210717 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9030933-1741-459f-8ba6-2fbca05eab03 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:28.211194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9030933-1741-459f-8ba6-2fbca05eab03 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:28.211764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9030933-1741-459f-8ba6-2fbca05eab03 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:28.213282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9030933-1741-459f-8ba6-2fbca05eab03 None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:28.214942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9030933-1741-459f-8ba6-2fbca05eab03 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:28.221712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 134/276] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:28 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:28.228792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9030933-1741-459f-8ba6-2fbca05eab03 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:28.293403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063fc2e-0032-493a-9408-bf6c60e74a08 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:28.313321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:28.329714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f9e4d-3227-4e64-9892-d099bd78c14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:28.330372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9e4d-3227-4e64-9892-d099bd78c14b None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:28.330656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9e4d-3227-4e64-9892-d099bd78c14b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:28.330931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9e4d-3227-4e64-9892-d099bd78c14b None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:28.332852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9030933-1741-459f-8ba6-2fbca05eab03 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:28.332945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f9e4d-3227-4e64-9892-d099bd78c14b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:28.333864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f9e4d-3227-4e64-9892-d099bd78c14b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:28.368266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fa303-f7b5-4c0c-a7d4-4652153d65a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:28.368885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fa303-f7b5-4c0c-a7d4-4652153d65a7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:28.369166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fa303-f7b5-4c0c-a7d4-4652153d65a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:28.369441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fa303-f7b5-4c0c-a7d4-4652153d65a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:28.370258 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fa303-f7b5-4c0c-a7d4-4652153d65a7 None admin] RBAC: Adding query filter params (role.id=36291c067d1949cfae0d704585f036c8, user.id=2f393ec159a4473eb4e0555cad6afaa2, scope.system=all) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:28.370572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fa303-f7b5-4c0c-a7d4-4652153d65a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:28.372000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fa303-f7b5-4c0c-a7d4-4652153d65a7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:28.377825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 135/277] 10.222.0.73 () {56 vars in 1308 bytes} [Thu Aug 4 21:21:28 2022] GET /identity/v3/role_assignments?user.id=2f393ec159a4473eb4e0555cad6afaa2&scope.system=all&role.id=36291c067d1949cfae0d704585f036c8 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:28.378553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fa303-f7b5-4c0c-a7d4-4652153d65a7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:28.466100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 143/278] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:21:28 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:28.466100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf5b7c-2d4c-42c0-b37e-742f5feda8c3 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:28.466100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf5b7c-2d4c-42c0-b37e-742f5feda8c3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:28.466100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf5b7c-2d4c-42c0-b37e-742f5feda8c3 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:28.466100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 144/279] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:28.473246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e426a-a78c-4bce-8780-9f62dee9e074 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:28.473567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e426a-a78c-4bce-8780-9f62dee9e074 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:28.473839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e426a-a78c-4bce-8780-9f62dee9e074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:28.515984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-106e426a-a78c-4bce-8780-9f62dee9e074 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:28.530038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-106e426a-a78c-4bce-8780-9f62dee9e074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:28.569703 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fa303-f7b5-4c0c-a7d4-4652153d65a7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:28.638656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 145/280] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:28.658526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e426a-a78c-4bce-8780-9f62dee9e074 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:28.659464 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd70eb0-ffbb-42fe-b415-6aff18f7723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:28.660100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd70eb0-ffbb-42fe-b415-6aff18f7723a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:28.660479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd70eb0-ffbb-42fe-b415-6aff18f7723a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:28.660749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd70eb0-ffbb-42fe-b415-6aff18f7723a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:28.661553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd70eb0-ffbb-42fe-b415-6aff18f7723a None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=d64e4c325db8448dadb785b8df946d6c, user.id=726553787fb341f485b8561971b880d1) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:28.661862 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd70eb0-ffbb-42fe-b415-6aff18f7723a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:28.663342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd70eb0-ffbb-42fe-b415-6aff18f7723a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:28.678650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 136/281] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:28 2022] GET /identity/v3/role_assignments?user.id=726553787fb341f485b8561971b880d1&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:28.680168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:28.789587 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e426a-a78c-4bce-8780-9f62dee9e074 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:28.808486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beed6c-9876-4292-bbf0-8eec54d71fc4 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:28.809012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beed6c-9876-4292-bbf0-8eec54d71fc4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:28.809290 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beed6c-9876-4292-bbf0-8eec54d71fc4 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:28.811095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 137/282] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:28.817240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a4102-fd15-4f63-8ddd-59cb55eaab4c None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:28.818080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a4102-fd15-4f63-8ddd-59cb55eaab4c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:28.818574 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a4102-fd15-4f63-8ddd-59cb55eaab4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:28.835293 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f295db-36b0-49b7-91f9-749aa33e90a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:28.835520 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f295db-36b0-49b7-91f9-749aa33e90a0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:28.835778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f295db-36b0-49b7-91f9-749aa33e90a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:28.836040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f295db-36b0-49b7-91f9-749aa33e90a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:28.842169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-8e3a4102-fd15-4f63-8ddd-59cb55eaab4c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:28.843742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8e3a4102-fd15-4f63-8ddd-59cb55eaab4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:28.846271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:28.855166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f295db-36b0-49b7-91f9-749aa33e90a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-57f295db-36b0-49b7-91f9-749aa33e90a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:28.862898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:28.863789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:28.864585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 146/283] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:28.871306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f295db-36b0-49b7-91f9-749aa33e90a0 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:28.937760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 138/284] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:28.980679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a4102-fd15-4f63-8ddd-59cb55eaab4c None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:28.989351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:29.022628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f295db-36b0-49b7-91f9-749aa33e90a0 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:29.063286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77168ae4-79c1-4dc9-b2a5-81a27cc60664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:29.064176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77168ae4-79c1-4dc9-b2a5-81a27cc60664 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:29.064612 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77168ae4-79c1-4dc9-b2a5-81a27cc60664 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:29.065023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77168ae4-79c1-4dc9-b2a5-81a27cc60664 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:29.066026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77168ae4-79c1-4dc9-b2a5-81a27cc60664 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:29.067041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77168ae4-79c1-4dc9-b2a5-81a27cc60664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:29.068731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77168ae4-79c1-4dc9-b2a5-81a27cc60664 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:29.077972 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 147/285] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:29.084920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77168ae4-79c1-4dc9-b2a5-81a27cc60664 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:29.126822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a4102-fd15-4f63-8ddd-59cb55eaab4c None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:29.175805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ec910-297a-4feb-9905-bc8607a9f46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:29.176693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ec910-297a-4feb-9905-bc8607a9f46c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:29.177137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ec910-297a-4feb-9905-bc8607a9f46c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:29.177547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ec910-297a-4feb-9905-bc8607a9f46c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:29.187836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:29.193358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ec910-297a-4feb-9905-bc8607a9f46c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-583ec910-297a-4feb-9905-bc8607a9f46c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:29.200297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:29.208861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:29.209871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 139/286] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:29.211664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ec910-297a-4feb-9905-bc8607a9f46c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:29.219518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77168ae4-79c1-4dc9-b2a5-81a27cc60664 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:29.274227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce85fe5-fd96-4c61-9671-0f552c527351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:29.275122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce85fe5-fd96-4c61-9671-0f552c527351 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:29.275573 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce85fe5-fd96-4c61-9671-0f552c527351 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:29.276002 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce85fe5-fd96-4c61-9671-0f552c527351 None admin] PATH_INFO: `/v3/users/8246203d30344a509f48280c9f8ff7a3` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:29.283052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce85fe5-fd96-4c61-9671-0f552c527351 None admin] RBAC: Authorizing `identity:get_user(user_id=8246203d30344a509f48280c9f8ff7a3)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:29.284668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce85fe5-fd96-4c61-9671-0f552c527351 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:29.286897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 148/287] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:29 2022] GET /identity/v3/users/8246203d30344a509f48280c9f8ff7a3 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:29.293384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce85fe5-fd96-4c61-9671-0f552c527351 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:29.301381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:29.351706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ec910-297a-4feb-9905-bc8607a9f46c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:29.416093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efdcc5-12c2-46d6-8ee1-c6553b50f810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:29.416808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efdcc5-12c2-46d6-8ee1-c6553b50f810 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:29.417206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efdcc5-12c2-46d6-8ee1-c6553b50f810 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:29.417484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efdcc5-12c2-46d6-8ee1-c6553b50f810 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:29.418288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56efdcc5-12c2-46d6-8ee1-c6553b50f810 None admin] RBAC: Adding query filter params (name=member) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:29.418598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56efdcc5-12c2-46d6-8ee1-c6553b50f810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:29.419989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56efdcc5-12c2-46d6-8ee1-c6553b50f810 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:29.437254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 140/288] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:29.446166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efdcc5-12c2-46d6-8ee1-c6553b50f810 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:29.480029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce85fe5-fd96-4c61-9671-0f552c527351 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:29.554360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:29.555391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e844ac-46ad-4207-ae59-16834d218a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:29.556048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e844ac-46ad-4207-ae59-16834d218a07 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:29.556335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e844ac-46ad-4207-ae59-16834d218a07 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:29.556613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e844ac-46ad-4207-ae59-16834d218a07 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:29.558354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e844ac-46ad-4207-ae59-16834d218a07 None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:29.559820 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e844ac-46ad-4207-ae59-16834d218a07 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:29.571163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 149/289] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:29 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:29.573047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e844ac-46ad-4207-ae59-16834d218a07 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:29.622346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efdcc5-12c2-46d6-8ee1-c6553b50f810 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:29.663125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ef67e-a44c-4c1d-bd7a-db80ecdb9ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:29.664433 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ef67e-a44c-4c1d-bd7a-db80ecdb9ef7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:29.664731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ef67e-a44c-4c1d-bd7a-db80ecdb9ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:29.665016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ef67e-a44c-4c1d-bd7a-db80ecdb9ef7 None admin] PATH_INFO: `/v3/users/f5e09ac9486143b583ec1b1150ccaf48` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:29.666322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ef67e-a44c-4c1d-bd7a-db80ecdb9ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=f5e09ac9486143b583ec1b1150ccaf48)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:29.667788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ef67e-a44c-4c1d-bd7a-db80ecdb9ef7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:29.674485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 141/290] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:29 2022] GET /identity/v3/users/f5e09ac9486143b583ec1b1150ccaf48 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:29.675885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ef67e-a44c-4c1d-bd7a-db80ecdb9ef7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:29.771894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:29.789510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ef67e-a44c-4c1d-bd7a-db80ecdb9ef7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:29.827144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc1280-3637-4dd6-b6d6-066ca5a8e769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:29.827775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc1280-3637-4dd6-b6d6-066ca5a8e769 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:29.828064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc1280-3637-4dd6-b6d6-066ca5a8e769 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:29.828346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc1280-3637-4dd6-b6d6-066ca5a8e769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:29.838531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:29.854854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e844ac-46ad-4207-ae59-16834d218a07 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:29.921015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72aa3d0-7a9a-4d6d-b79c-c601e5777c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:29.921623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aa3d0-7a9a-4d6d-b79c-c601e5777c37 None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:29.921907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aa3d0-7a9a-4d6d-b79c-c601e5777c37 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:29.922209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aa3d0-7a9a-4d6d-b79c-c601e5777c37 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c/users/8246203d30344a509f48280c9f8ff7a3/roles/2ca1bca713f745ea9b40dc20cb5fa2a1` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:29.927135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72aa3d0-7a9a-4d6d-b79c-c601e5777c37 None admin] RBAC: Authorizing `identity:create_grant(project_id=d64e4c325db8448dadb785b8df946d6c, user_id=8246203d30344a509f48280c9f8ff7a3, role_id=2ca1bca713f745ea9b40dc20cb5fa2a1)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:29.928667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72aa3d0-7a9a-4d6d-b79c-c601e5777c37 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:29.949669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc1280-3637-4dd6-b6d6-066ca5a8e769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:29.951177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc1280-3637-4dd6-b6d6-066ca5a8e769 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:30.046238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 150/291] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:21:29 2022] PUT /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c/users/8246203d30344a509f48280c9f8ff7a3/roles/2ca1bca713f745ea9b40dc20cb5fa2a1 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:30.080434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 142/292] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:30.087255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72aa3d0-7a9a-4d6d-b79c-c601e5777c37 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:30.240846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72aa3d0-7a9a-4d6d-b79c-c601e5777c37 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:30.273182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:30.278904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc781ae8-0b7d-40dd-9def-a6a9c7ad3be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:30.279552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc781ae8-0b7d-40dd-9def-a6a9c7ad3be9 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:30.279839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc781ae8-0b7d-40dd-9def-a6a9c7ad3be9 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:30.280116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc781ae8-0b7d-40dd-9def-a6a9c7ad3be9 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:30.281374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc781ae8-0b7d-40dd-9def-a6a9c7ad3be9 None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:30.282865 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc781ae8-0b7d-40dd-9def-a6a9c7ad3be9 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:30.284622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 151/293] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:30 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:30.291071 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc1280-3637-4dd6-b6d6-066ca5a8e769 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:30.325737 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:30.403802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc1280-3637-4dd6-b6d6-066ca5a8e769 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:30.451615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835212e9-b928-4475-95b0-c7da7d42d73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:30.452238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835212e9-b928-4475-95b0-c7da7d42d73e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:30.452517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835212e9-b928-4475-95b0-c7da7d42d73e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:30.452791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835212e9-b928-4475-95b0-c7da7d42d73e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:30.454553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835212e9-b928-4475-95b0-c7da7d42d73e None admin] RBAC: Adding query filter params (role.id=36291c067d1949cfae0d704585f036c8, scope.project.id=d64e4c325db8448dadb785b8df946d6c, user.id=f5e09ac9486143b583ec1b1150ccaf48) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:30.454864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835212e9-b928-4475-95b0-c7da7d42d73e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:30.456374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835212e9-b928-4475-95b0-c7da7d42d73e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:30.466507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 143/294] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:30 2022] GET /identity/v3/role_assignments?user.id=f5e09ac9486143b583ec1b1150ccaf48&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=36291c067d1949cfae0d704585f036c8 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:30.743404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9cc98-7f7c-4c86-abd7-7ea79cb6dc19 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:30.744089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9cc98-7f7c-4c86-abd7-7ea79cb6dc19 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:30.744578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9cc98-7f7c-4c86-abd7-7ea79cb6dc19 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:30.746443 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 152/295] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:30.752027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ea3fa-6f20-4f9b-9b2e-7d3b5d1e9c75 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:30.752583 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ea3fa-6f20-4f9b-9b2e-7d3b5d1e9c75 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:30.753060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ea3fa-6f20-4f9b-9b2e-7d3b5d1e9c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:30.773374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-a78ea3fa-6f20-4f9b-9b2e-7d3b5d1e9c75 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:30.774919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a78ea3fa-6f20-4f9b-9b2e-7d3b5d1e9c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:30.849689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 144/296] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:30.872639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9cc98-7f7c-4c86-abd7-7ea79cb6dc19 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:30.984190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9cc98-7f7c-4c86-abd7-7ea79cb6dc19 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:31.024926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c018b-578b-499e-80ce-4f8b548e3b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:31.026043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c018b-578b-499e-80ce-4f8b548e3b00 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.026529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c018b-578b-499e-80ce-4f8b548e3b00 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.027054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c018b-578b-499e-80ce-4f8b548e3b00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.038775 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c018b-578b-499e-80ce-4f8b548e3b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-800c018b-578b-499e-80ce-4f8b548e3b00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:31.044695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:31.045574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:31.046426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:31.048816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 153/297] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:31.055124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ea3fa-6f20-4f9b-9b2e-7d3b5d1e9c75 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:31.085700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:31.172270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7af5f-dafe-4632-a346-329e8aac403c None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.172591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7af5f-dafe-4632-a346-329e8aac403c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.173082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7af5f-dafe-4632-a346-329e8aac403c None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.175028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 154/298] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:21:31.181858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd229879-73b4-4e05-adb2-d94c2cacbed0 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.182182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd229879-73b4-4e05-adb2-d94c2cacbed0 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.182457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd229879-73b4-4e05-adb2-d94c2cacbed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.188195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ea3fa-6f20-4f9b-9b2e-7d3b5d1e9c75 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:31.200859 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-fd229879-73b4-4e05-adb2-d94c2cacbed0 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:31.202143 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-fd229879-73b4-4e05-adb2-d94c2cacbed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:31.224924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d031e86-cdc0-49dc-bcd1-5ba090c98598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:31.225894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d031e86-cdc0-49dc-bcd1-5ba090c98598 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.226409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d031e86-cdc0-49dc-bcd1-5ba090c98598 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.226954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d031e86-cdc0-49dc-bcd1-5ba090c98598 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.228017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d031e86-cdc0-49dc-bcd1-5ba090c98598 None admin] RBAC: Adding query filter params (name=member) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:31.228522 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d031e86-cdc0-49dc-bcd1-5ba090c98598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:31.230323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d031e86-cdc0-49dc-bcd1-5ba090c98598 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:31.244998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 145/299] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:31.246959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d031e86-cdc0-49dc-bcd1-5ba090c98598 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:31.261636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:31.273185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 155/300] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:31.296831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd229879-73b4-4e05-adb2-d94c2cacbed0 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:31.350373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:31.354554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d031e86-cdc0-49dc-bcd1-5ba090c98598 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:31.391126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00970c6a-2e36-466b-bc3f-7e3457b4bff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:31.392161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00970c6a-2e36-466b-bc3f-7e3457b4bff1 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.392646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00970c6a-2e36-466b-bc3f-7e3457b4bff1 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.393141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00970c6a-2e36-466b-bc3f-7e3457b4bff1 None admin] PATH_INFO: `/v3/users/2f393ec159a4473eb4e0555cad6afaa2` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.395093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00970c6a-2e36-466b-bc3f-7e3457b4bff1 None admin] RBAC: Authorizing `identity:get_user(user_id=2f393ec159a4473eb4e0555cad6afaa2)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:31.396788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00970c6a-2e36-466b-bc3f-7e3457b4bff1 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:31.403286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd229879-73b4-4e05-adb2-d94c2cacbed0 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:31.403577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 146/301] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:31 2022] GET /identity/v3/users/2f393ec159a4473eb4e0555cad6afaa2 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:31.404895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00970c6a-2e36-466b-bc3f-7e3457b4bff1 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:31.438956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4149d-c797-47aa-8866-cae4ccbf1de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:31.439625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4149d-c797-47aa-8866-cae4ccbf1de4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.439851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4149d-c797-47aa-8866-cae4ccbf1de4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.440122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4149d-c797-47aa-8866-cae4ccbf1de4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.451267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4149d-c797-47aa-8866-cae4ccbf1de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-81c4149d-c797-47aa-8866-cae4ccbf1de4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:31.456289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:31.457741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:31.459130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:31.462085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 156/302] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:31.464065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4149d-c797-47aa-8866-cae4ccbf1de4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:31.522524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:31.536510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00970c6a-2e36-466b-bc3f-7e3457b4bff1 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:31.573077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd2c7c-47f6-4ff2-a56b-1eacf2231661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:31.573679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4149d-c797-47aa-8866-cae4ccbf1de4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:31.573786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd2c7c-47f6-4ff2-a56b-1eacf2231661 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.573958 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd2c7c-47f6-4ff2-a56b-1eacf2231661 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.574254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd2c7c-47f6-4ff2-a56b-1eacf2231661 None admin] PATH_INFO: `/v3/system/users/2f393ec159a4473eb4e0555cad6afaa2/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.575698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd2c7c-47f6-4ff2-a56b-1eacf2231661 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2f393ec159a4473eb4e0555cad6afaa2, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:31.577058 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd2c7c-47f6-4ff2-a56b-1eacf2231661 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:31.609370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9264e-c590-4b4e-81e0-b9cd931813af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:31.610266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9264e-c590-4b4e-81e0-b9cd931813af None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.610547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9264e-c590-4b4e-81e0-b9cd931813af None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.610898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9264e-c590-4b4e-81e0-b9cd931813af None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.611621 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9264e-c590-4b4e-81e0-b9cd931813af None admin] RBAC: Adding query filter params (name=reader) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:31.611908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9264e-c590-4b4e-81e0-b9cd931813af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:31.613440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9264e-c590-4b4e-81e0-b9cd931813af None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:31.621340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 157/303] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:31.622562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8955d-7ee3-43e4-8b9c-6300983b2dbd None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.622847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8955d-7ee3-43e4-8b9c-6300983b2dbd None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.623119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8955d-7ee3-43e4-8b9c-6300983b2dbd None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.624475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 158/304] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:31.626680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8955d-7ee3-43e4-8b9c-6300983b2dbd None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:31.677620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 147/305] 10.222.0.73 () {58 vars in 1334 bytes} [Thu Aug 4 21:21:31 2022] PUT /identity/v3/system/users/2f393ec159a4473eb4e0555cad6afaa2/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:21:31.686234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0398da-ede0-4494-81b1-69eb616d86bd None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.686234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0398da-ede0-4494-81b1-69eb616d86bd None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.686234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0398da-ede0-4494-81b1-69eb616d86bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.709819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-9e0398da-ede0-4494-81b1-69eb616d86bd None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:31.711222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-9e0398da-ede0-4494-81b1-69eb616d86bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:31.727361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:31.746966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8955d-7ee3-43e4-8b9c-6300983b2dbd None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:31.786389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:31.790052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca47d8-e478-4c4e-9461-fc9339ceed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:31.791017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca47d8-e478-4c4e-9461-fc9339ceed96 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.791374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca47d8-e478-4c4e-9461-fc9339ceed96 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.791747 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca47d8-e478-4c4e-9461-fc9339ceed96 None admin] PATH_INFO: `/v3/users/d0442007c633432ea0a73ce8bfee726a` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.812942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 148/306] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:31.822891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca47d8-e478-4c4e-9461-fc9339ceed96 None admin] RBAC: Authorizing `identity:get_user(user_id=d0442007c633432ea0a73ce8bfee726a)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:31.824759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca47d8-e478-4c4e-9461-fc9339ceed96 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:31.826917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 159/307] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:31 2022] GET /identity/v3/users/d0442007c633432ea0a73ce8bfee726a => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:31.833807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0398da-ede0-4494-81b1-69eb616d86bd None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:31.841060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca47d8-e478-4c4e-9461-fc9339ceed96 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:31.951981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0398da-ede0-4494-81b1-69eb616d86bd None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:31.956562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca47d8-e478-4c4e-9461-fc9339ceed96 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:31.994257 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca11447-4cec-4593-a927-e5b5c62fa461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:31.995098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca11447-4cec-4593-a927-e5b5c62fa461 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.995462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca11447-4cec-4593-a927-e5b5c62fa461 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.995741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca11447-4cec-4593-a927-e5b5c62fa461 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.996052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca637642-145f-42b1-a665-d358a3c43fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:31.996790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca637642-145f-42b1-a665-d358a3c43fa9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:31.996943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca11447-4cec-4593-a927-e5b5c62fa461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:31.997187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca637642-145f-42b1-a665-d358a3c43fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:31.997523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca637642-145f-42b1-a665-d358a3c43fa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:31.998411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca11447-4cec-4593-a927-e5b5c62fa461 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:31.998518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca637642-145f-42b1-a665-d358a3c43fa9 None admin] RBAC: Adding query filter params (role.id=2ca1bca713f745ea9b40dc20cb5fa2a1, user.id=d0442007c633432ea0a73ce8bfee726a, scope.system=all) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:31.998718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca637642-145f-42b1-a665-d358a3c43fa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:32.000368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 160/308] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:32.000478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca637642-145f-42b1-a665-d358a3c43fa9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:32.006589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 149/309] 10.222.0.73 () {56 vars in 1308 bytes} [Thu Aug 4 21:21:31 2022] GET /identity/v3/role_assignments?user.id=d0442007c633432ea0a73ce8bfee726a&scope.system=all&role.id=2ca1bca713f745ea9b40dc20cb5fa2a1 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:32.007167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca11447-4cec-4593-a927-e5b5c62fa461 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:32.062836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:32.118767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca11447-4cec-4593-a927-e5b5c62fa461 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:32.159987 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf702e-d463-423d-9fdb-a7b9bdf3eebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:32.161640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf702e-d463-423d-9fdb-a7b9bdf3eebf None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:32.161923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf702e-d463-423d-9fdb-a7b9bdf3eebf None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:32.162224 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf702e-d463-423d-9fdb-a7b9bdf3eebf None admin] PATH_INFO: `/v3/groups` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:32.162893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf702e-d463-423d-9fdb-a7b9bdf3eebf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:32.164340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf702e-d463-423d-9fdb-a7b9bdf3eebf None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:32.256277 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 161/310] 10.222.0.73 () {60 vars in 1134 bytes} [Thu Aug 4 21:21:32 2022] POST /identity/v3/groups => generated 229 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:32.461687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5816aaf-595d-4e0b-aba8-cc472ae224b4 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:32.462219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5816aaf-595d-4e0b-aba8-cc472ae224b4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:32.462292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5816aaf-595d-4e0b-aba8-cc472ae224b4 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:32.463755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 150/311] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:32.470223 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625b3b2-16b3-4a38-821b-03b739a44db3 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:32.470679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625b3b2-16b3-4a38-821b-03b739a44db3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:32.470956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625b3b2-16b3-4a38-821b-03b739a44db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:32.490316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-2625b3b2-16b3-4a38-821b-03b739a44db3 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:32.491517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2625b3b2-16b3-4a38-821b-03b739a44db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:32.560619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 162/312] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:32.589163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5816aaf-595d-4e0b-aba8-cc472ae224b4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:32.603867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:32.696942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5816aaf-595d-4e0b-aba8-cc472ae224b4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:32.732532 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f6737-f774-4da0-b108-8d4760959b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:32.733086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f6737-f774-4da0-b108-8d4760959b28 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:32.733365 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f6737-f774-4da0-b108-8d4760959b28 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:32.733632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f6737-f774-4da0-b108-8d4760959b28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:32.745204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f6737-f774-4da0-b108-8d4760959b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-596f6737-f774-4da0-b108-8d4760959b28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:32.750186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:32.751080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:32.751877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 151/313] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:32.757880 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625b3b2-16b3-4a38-821b-03b739a44db3 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:32.794451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:32.862448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625b3b2-16b3-4a38-821b-03b739a44db3 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:32.901576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2686bc-7f02-439c-b9c0-0cb4a8cdf85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:32.902250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2686bc-7f02-439c-b9c0-0cb4a8cdf85b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:32.902539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2686bc-7f02-439c-b9c0-0cb4a8cdf85b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:32.902817 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2686bc-7f02-439c-b9c0-0cb4a8cdf85b None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:32.903533 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2686bc-7f02-439c-b9c0-0cb4a8cdf85b None admin] RBAC: Adding query filter params (name=reader) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:32.903843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2686bc-7f02-439c-b9c0-0cb4a8cdf85b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:32.905232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2686bc-7f02-439c-b9c0-0cb4a8cdf85b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:32.914151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 163/314] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:32.919619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f6737-f774-4da0-b108-8d4760959b28 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:32.991381 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:33.022395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f6737-f774-4da0-b108-8d4760959b28 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:33.057836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a17fb4-6038-4916-8840-5a2678f65ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:33.058528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a17fb4-6038-4916-8840-5a2678f65ed8 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:33.058813 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a17fb4-6038-4916-8840-5a2678f65ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:33.059089 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a17fb4-6038-4916-8840-5a2678f65ed8 None admin] PATH_INFO: `/v3/users/8246203d30344a509f48280c9f8ff7a3` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:33.060342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a17fb4-6038-4916-8840-5a2678f65ed8 None admin] RBAC: Authorizing `identity:get_user(user_id=8246203d30344a509f48280c9f8ff7a3)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:33.061755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a17fb4-6038-4916-8840-5a2678f65ed8 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:33.063362 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 152/315] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:32 2022] GET /identity/v3/users/8246203d30344a509f48280c9f8ff7a3 => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:33.069540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2686bc-7f02-439c-b9c0-0cb4a8cdf85b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:33.166298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:33.176357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2686bc-7f02-439c-b9c0-0cb4a8cdf85b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:33.225553 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05094b-05b7-481a-ba51-065b48228a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:33.226214 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05094b-05b7-481a-ba51-065b48228a7f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:33.226503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05094b-05b7-481a-ba51-065b48228a7f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:33.226786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05094b-05b7-481a-ba51-065b48228a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:33.309850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05094b-05b7-481a-ba51-065b48228a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:33.311734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05094b-05b7-481a-ba51-065b48228a7f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:33.405599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:33.420710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 164/316] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:33.427120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a17fb4-6038-4916-8840-5a2678f65ed8 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:33.534436 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a17fb4-6038-4916-8840-5a2678f65ed8 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:33.570164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a651c-4237-4f7b-8720-c3dd41e1054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:33.570737 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a651c-4237-4f7b-8720-c3dd41e1054b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:33.571017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a651c-4237-4f7b-8720-c3dd41e1054b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:33.571295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a651c-4237-4f7b-8720-c3dd41e1054b None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:33.572491 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a651c-4237-4f7b-8720-c3dd41e1054b None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:33.573928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a651c-4237-4f7b-8720-c3dd41e1054b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:33.575937 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 153/317] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:33 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:33.581413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05094b-05b7-481a-ba51-065b48228a7f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:33.686775 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05094b-05b7-481a-ba51-065b48228a7f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:33.723119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d20eb2-0df5-4c39-97c9-b32021be4ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:33.723891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d20eb2-0df5-4c39-97c9-b32021be4ec0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:33.724177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d20eb2-0df5-4c39-97c9-b32021be4ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:33.724455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d20eb2-0df5-4c39-97c9-b32021be4ec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:33.725237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d20eb2-0df5-4c39-97c9-b32021be4ec0 None admin] RBAC: Adding query filter params (role.id=2ca1bca713f745ea9b40dc20cb5fa2a1, scope.project.id=d64e4c325db8448dadb785b8df946d6c, user.id=8246203d30344a509f48280c9f8ff7a3) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:33.725547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d20eb2-0df5-4c39-97c9-b32021be4ec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:33.727001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d20eb2-0df5-4c39-97c9-b32021be4ec0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:33.733380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 165/318] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:21:33 2022] GET /identity/v3/role_assignments?user.id=8246203d30344a509f48280c9f8ff7a3&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=2ca1bca713f745ea9b40dc20cb5fa2a1 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:33.833190 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9686746-45ad-40d1-80ca-2e1f1d7b9907 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:33.833805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9686746-45ad-40d1-80ca-2e1f1d7b9907 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:33.834339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9686746-45ad-40d1-80ca-2e1f1d7b9907 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:33.835917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 154/319] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:33.841885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c58f-b818-41e8-ac96-572369aac6d3 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:33.842515 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c58f-b818-41e8-ac96-572369aac6d3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:33.842991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c58f-b818-41e8-ac96-572369aac6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:33.863949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-11c6c58f-b818-41e8-ac96-572369aac6d3 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:33.865762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-11c6c58f-b818-41e8-ac96-572369aac6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:33.922175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:33.940604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 166/320] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:33.964873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9686746-45ad-40d1-80ca-2e1f1d7b9907 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:33.976179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:34.073514 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9686746-45ad-40d1-80ca-2e1f1d7b9907 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:34.107942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16616bad-43d2-4614-84b7-f92228119651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:34.108503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16616bad-43d2-4614-84b7-f92228119651 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.108783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16616bad-43d2-4614-84b7-f92228119651 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.109051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16616bad-43d2-4614-84b7-f92228119651 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.119852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16616bad-43d2-4614-84b7-f92228119651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-16616bad-43d2-4614-84b7-f92228119651 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:34.124846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:34.125716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:34.126978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 155/321] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:34.132473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6c58f-b818-41e8-ac96-572369aac6d3 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:34.137864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e82ca-8af8-46b3-8d91-d06fb11af4bf None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.138184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e82ca-8af8-46b3-8d91-d06fb11af4bf None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.138461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e82ca-8af8-46b3-8d91-d06fb11af4bf None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.139935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 156/322] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:21:34.145189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790aecaf-e992-4f96-8f68-19771197de3b None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.145475 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790aecaf-e992-4f96-8f68-19771197de3b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.145746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790aecaf-e992-4f96-8f68-19771197de3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.165380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-790aecaf-e992-4f96-8f68-19771197de3b None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:34.166605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-790aecaf-e992-4f96-8f68-19771197de3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:34.228440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:34.238048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 157/323] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:34.247346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6c58f-b818-41e8-ac96-572369aac6d3 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:34.258384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790aecaf-e992-4f96-8f68-19771197de3b None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:34.284851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b738496-5f6e-46e3-b41e-906c5c761d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:34.285645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b738496-5f6e-46e3-b41e-906c5c761d14 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.285926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b738496-5f6e-46e3-b41e-906c5c761d14 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.286290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b738496-5f6e-46e3-b41e-906c5c761d14 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.286940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b738496-5f6e-46e3-b41e-906c5c761d14 None admin] RBAC: Adding query filter params (name=member) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:34.287243 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b738496-5f6e-46e3-b41e-906c5c761d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:34.288629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b738496-5f6e-46e3-b41e-906c5c761d14 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:34.297872 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 167/324] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:34.303620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b738496-5f6e-46e3-b41e-906c5c761d14 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:34.353188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:34.366238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790aecaf-e992-4f96-8f68-19771197de3b None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:34.403328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b0461-34ec-47a7-b835-f84c17157b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:34.404489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b0461-34ec-47a7-b835-f84c17157b87 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.404902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b0461-34ec-47a7-b835-f84c17157b87 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.405185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b0461-34ec-47a7-b835-f84c17157b87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.410484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b738496-5f6e-46e3-b41e-906c5c761d14 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:34.419519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b0461-34ec-47a7-b835-f84c17157b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-d61b0461-34ec-47a7-b835-f84c17157b87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:34.427078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:34.427971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:34.428766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:34.433024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 158/325] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:34.433774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a9239-1889-4222-8f39-1b5860b4698b None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.434087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a9239-1889-4222-8f39-1b5860b4698b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.434365 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a9239-1889-4222-8f39-1b5860b4698b None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.435643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 159/326] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:21:34.436333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a9239-1889-4222-8f39-1b5860b4698b None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:34.452613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cde3a-bd6c-4441-aff2-c22058570fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:34.453261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cde3a-bd6c-4441-aff2-c22058570fda None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.453550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cde3a-bd6c-4441-aff2-c22058570fda None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.453822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cde3a-bd6c-4441-aff2-c22058570fda None admin] PATH_INFO: `/v3/users/2f393ec159a4473eb4e0555cad6afaa2` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.455039 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cde3a-bd6c-4441-aff2-c22058570fda None admin] RBAC: Authorizing `identity:get_user(user_id=2f393ec159a4473eb4e0555cad6afaa2)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:34.456567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cde3a-bd6c-4441-aff2-c22058570fda None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:34.458524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 168/327] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:34 2022] GET /identity/v3/users/2f393ec159a4473eb4e0555cad6afaa2 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:34.459637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3dfd-7837-4039-977a-ab3bc80ed69f None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.459932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3dfd-7837-4039-977a-ab3bc80ed69f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.460207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3dfd-7837-4039-977a-ab3bc80ed69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.479688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d59a3dfd-7837-4039-977a-ab3bc80ed69f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:34.480886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d59a3dfd-7837-4039-977a-ab3bc80ed69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:34.499788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:34.547302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a9239-1889-4222-8f39-1b5860b4698b None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:34.549254 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 169/328] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:34.550251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a3dfd-7837-4039-977a-ab3bc80ed69f None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:34.584610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691c290-96ee-4baf-8d13-7f0a740ac1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:34.585225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691c290-96ee-4baf-8d13-7f0a740ac1bf None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.585510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691c290-96ee-4baf-8d13-7f0a740ac1bf None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.585781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691c290-96ee-4baf-8d13-7f0a740ac1bf None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.586528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691c290-96ee-4baf-8d13-7f0a740ac1bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:34.586838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691c290-96ee-4baf-8d13-7f0a740ac1bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:34.588273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691c290-96ee-4baf-8d13-7f0a740ac1bf None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:34.596305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 160/329] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:34.600318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691c290-96ee-4baf-8d13-7f0a740ac1bf None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:34.602612 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:34.658909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a3dfd-7837-4039-977a-ab3bc80ed69f None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:34.692869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49d3aa-7d9b-4cd4-a3f8-07b5e415beb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:34.693474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49d3aa-7d9b-4cd4-a3f8-07b5e415beb5 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.693758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49d3aa-7d9b-4cd4-a3f8-07b5e415beb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.694051 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49d3aa-7d9b-4cd4-a3f8-07b5e415beb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.694939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49d3aa-7d9b-4cd4-a3f8-07b5e415beb5 None admin] RBAC: Adding query filter params (role.id=36291c067d1949cfae0d704585f036c8, user.id=2f393ec159a4473eb4e0555cad6afaa2, scope.system=all) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:34.695245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49d3aa-7d9b-4cd4-a3f8-07b5e415beb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:34.696679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49d3aa-7d9b-4cd4-a3f8-07b5e415beb5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:34.700102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691c290-96ee-4baf-8d13-7f0a740ac1bf None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:34.707781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 170/330] 10.222.0.73 () {56 vars in 1308 bytes} [Thu Aug 4 21:21:34 2022] GET /identity/v3/role_assignments?user.id=2f393ec159a4473eb4e0555cad6afaa2&scope.system=all&role.id=36291c067d1949cfae0d704585f036c8 => generated 510 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:34.708576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49d3aa-7d9b-4cd4-a3f8-07b5e415beb5 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:34.719151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:34.741991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba9257-b8b1-4b05-9d92-8cf007654294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:34.742987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba9257-b8b1-4b05-9d92-8cf007654294 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.743272 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba9257-b8b1-4b05-9d92-8cf007654294 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.743555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba9257-b8b1-4b05-9d92-8cf007654294 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.744804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba9257-b8b1-4b05-9d92-8cf007654294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:34.746331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba9257-b8b1-4b05-9d92-8cf007654294 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:34.748021 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 161/331] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:34.753942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba9257-b8b1-4b05-9d92-8cf007654294 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:34.829708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49d3aa-7d9b-4cd4-a3f8-07b5e415beb5 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:34.831556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:34.868161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a5633-bee9-46ef-87c2-7ba81c360467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:34.868806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a5633-bee9-46ef-87c2-7ba81c360467 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.869086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a5633-bee9-46ef-87c2-7ba81c360467 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.869358 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a5633-bee9-46ef-87c2-7ba81c360467 None admin] PATH_INFO: `/v3/users/d0442007c633432ea0a73ce8bfee726a` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.870554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a5633-bee9-46ef-87c2-7ba81c360467 None admin] RBAC: Authorizing `identity:get_user(user_id=d0442007c633432ea0a73ce8bfee726a)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:34.871998 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a5633-bee9-46ef-87c2-7ba81c360467 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:34.873779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 171/332] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:34 2022] GET /identity/v3/users/d0442007c633432ea0a73ce8bfee726a => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:34.878651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba9257-b8b1-4b05-9d92-8cf007654294 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:34.879158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a5633-bee9-46ef-87c2-7ba81c360467 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:34.922440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d3c44-ba3c-4a65-9179-e6d6e7154147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:34.922834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d3c44-ba3c-4a65-9179-e6d6e7154147 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:34.923115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d3c44-ba3c-4a65-9179-e6d6e7154147 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:34.923385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d3c44-ba3c-4a65-9179-e6d6e7154147 None admin] PATH_INFO: `/v3/groups` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:34.924051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d3c44-ba3c-4a65-9179-e6d6e7154147 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:34.925499 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d3c44-ba3c-4a65-9179-e6d6e7154147 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:34.981605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a5633-bee9-46ef-87c2-7ba81c360467 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:35.014403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70da71-7aba-433c-81ee-56524d84f29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:35.015052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70da71-7aba-433c-81ee-56524d84f29b None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:35.015347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70da71-7aba-433c-81ee-56524d84f29b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:35.015602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70da71-7aba-433c-81ee-56524d84f29b None admin] PATH_INFO: `/v3/system/users/d0442007c633432ea0a73ce8bfee726a/roles/2ca1bca713f745ea9b40dc20cb5fa2a1` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:35.017020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70da71-7aba-433c-81ee-56524d84f29b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d0442007c633432ea0a73ce8bfee726a, role_id=2ca1bca713f745ea9b40dc20cb5fa2a1)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:35.018424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70da71-7aba-433c-81ee-56524d84f29b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:35.066013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 162/333] 10.222.0.73 () {60 vars in 1134 bytes} [Thu Aug 4 21:21:34 2022] POST /identity/v3/groups => generated 226 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:21:35.101642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 172/334] 10.222.0.73 () {58 vars in 1334 bytes} [Thu Aug 4 21:21:34 2022] PUT /identity/v3/system/users/d0442007c633432ea0a73ce8bfee726a/roles/2ca1bca713f745ea9b40dc20cb5fa2a1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:21:37.580914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaf4bb-ba56-4f16-a172-f637146bdd38 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:37.582572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaf4bb-ba56-4f16-a172-f637146bdd38 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:37.583067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaf4bb-ba56-4f16-a172-f637146bdd38 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:37.584917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 163/335] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:37.589979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b5845-bad8-4e09-90e4-f79a14c00d27 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:37.590886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b5845-bad8-4e09-90e4-f79a14c00d27 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:37.591380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b5845-bad8-4e09-90e4-f79a14c00d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:37.617071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-f71b5845-bad8-4e09-90e4-f79a14c00d27 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:37.618659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f71b5845-bad8-4e09-90e4-f79a14c00d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:37.650591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:37.712336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 173/336] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:37.752065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaf4bb-ba56-4f16-a172-f637146bdd38 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:37.814242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:37.880623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaf4bb-ba56-4f16-a172-f637146bdd38 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:37.924725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3adf3-f918-4039-8bb7-29c38cd5dfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:37.926394 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3adf3-f918-4039-8bb7-29c38cd5dfb3 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:37.926949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3adf3-f918-4039-8bb7-29c38cd5dfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:37.927423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3adf3-f918-4039-8bb7-29c38cd5dfb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:37.946622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3adf3-f918-4039-8bb7-29c38cd5dfb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-0cc3adf3-f918-4039-8bb7-29c38cd5dfb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:37.952538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:37.953442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 21:21:37.954281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:37.956990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 164/337] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:37.967121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b5845-bad8-4e09-90e4-f79a14c00d27 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:37.989804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af6742-fd4e-41fa-a5b3-a9a8ffe00aa5 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:37.990450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af6742-fd4e-41fa-a5b3-a9a8ffe00aa5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:37.990932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af6742-fd4e-41fa-a5b3-a9a8ffe00aa5 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:37.992502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 165/338] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:37.997912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8759106-276f-476d-8fa5-d96b74ea78ad None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:37.998454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8759106-276f-476d-8fa5-d96b74ea78ad None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:37.998933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8759106-276f-476d-8fa5-d96b74ea78ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:38.050114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c8759106-276f-476d-8fa5-d96b74ea78ad None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:38.057222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c8759106-276f-476d-8fa5-d96b74ea78ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:38.185377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 166/339] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:38.185377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2650046-f861-4937-a8c6-81bc36e65653 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:38.185377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2650046-f861-4937-a8c6-81bc36e65653 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:38.185377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2650046-f861-4937-a8c6-81bc36e65653 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:38.194502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 167/340] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:38 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:38.197953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffa23d-d4e5-4965-9a33-08031ed9c9ba None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:38.198519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffa23d-d4e5-4965-9a33-08031ed9c9ba None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:38.199158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffa23d-d4e5-4965-9a33-08031ed9c9ba None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:38.201056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 168/341] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:38.202624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f7f0b-8681-4f90-906e-40c551540721 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:38.203133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f7f0b-8681-4f90-906e-40c551540721 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:38.203606 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f7f0b-8681-4f90-906e-40c551540721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:38.226338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-950f7f0b-8681-4f90-906e-40c551540721 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:38.228385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-950f7f0b-8681-4f90-906e-40c551540721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:38.243000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b5845-bad8-4e09-90e4-f79a14c00d27 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:38.274792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:38.280331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bb68f-1107-4b20-ba86-9a65544a4ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:38.282600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bb68f-1107-4b20-ba86-9a65544a4ed1 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:38.282600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bb68f-1107-4b20-ba86-9a65544a4ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:38.282600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bb68f-1107-4b20-ba86-9a65544a4ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:38.282600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bb68f-1107-4b20-ba86-9a65544a4ed1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:38.282907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bb68f-1107-4b20-ba86-9a65544a4ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:38.284937 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bb68f-1107-4b20-ba86-9a65544a4ed1 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:38.299427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 174/342] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:38.301282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146250fb-528e-4452-8525-c86ae09a5bcb None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:38.301572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146250fb-528e-4452-8525-c86ae09a5bcb None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:38.301851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146250fb-528e-4452-8525-c86ae09a5bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:38.330630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 169/343] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:38.331489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f7f0b-8681-4f90-906e-40c551540721 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:38.333130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-146250fb-528e-4452-8525-c86ae09a5bcb None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:38.334704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-146250fb-528e-4452-8525-c86ae09a5bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:38.367251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:38.427713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 175/344] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:38.428564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146250fb-528e-4452-8525-c86ae09a5bcb None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:38.467313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f7f0b-8681-4f90-906e-40c551540721 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:38.516720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0963ec-3b3f-4d1a-b7cb-7b1ea120a5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:38.517974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0963ec-3b3f-4d1a-b7cb-7b1ea120a5c8 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:38.518461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0963ec-3b3f-4d1a-b7cb-7b1ea120a5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:38.519151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0963ec-3b3f-4d1a-b7cb-7b1ea120a5c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:38.536951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0963ec-3b3f-4d1a-b7cb-7b1ea120a5c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-5d0963ec-3b3f-4d1a-b7cb-7b1ea120a5c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:38.543474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:38.544427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:38.545228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 170/345] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:38.546666 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0963ec-3b3f-4d1a-b7cb-7b1ea120a5c8 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:38.572727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146250fb-528e-4452-8525-c86ae09a5bcb None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:38.581485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:38.639882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0df5f-344c-49a3-b708-8aa8bf7b40bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:38.640562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0df5f-344c-49a3-b708-8aa8bf7b40bf None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:38.640851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0df5f-344c-49a3-b708-8aa8bf7b40bf None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:38.641131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0df5f-344c-49a3-b708-8aa8bf7b40bf None admin] PATH_INFO: `/v3/users/d0442007c633432ea0a73ce8bfee726a` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:38.642370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0df5f-344c-49a3-b708-8aa8bf7b40bf None admin] RBAC: Authorizing `identity:get_user(user_id=d0442007c633432ea0a73ce8bfee726a)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:38.643816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0df5f-344c-49a3-b708-8aa8bf7b40bf None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:38.647068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 176/346] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:21:38 2022] GET /identity/v3/users/d0442007c633432ea0a73ce8bfee726a => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:38.648326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0df5f-344c-49a3-b708-8aa8bf7b40bf None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:38.679390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:38.682238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0963ec-3b3f-4d1a-b7cb-7b1ea120a5c8 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:38.727817 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77706c-ccaf-4983-9548-85551e15e135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:38.728485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77706c-ccaf-4983-9548-85551e15e135 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:38.728863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77706c-ccaf-4983-9548-85551e15e135 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:38.729142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77706c-ccaf-4983-9548-85551e15e135 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:38.742157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c77706c-ccaf-4983-9548-85551e15e135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-0c77706c-ccaf-4983-9548-85551e15e135 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:38.749255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:38.751750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 171/347] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:38.754672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77706c-ccaf-4983-9548-85551e15e135 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:38.776735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0df5f-344c-49a3-b708-8aa8bf7b40bf None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:38.827068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64336595-a9c5-4c94-836c-378a0206c051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:38.827713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64336595-a9c5-4c94-836c-378a0206c051 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:38.827992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64336595-a9c5-4c94-836c-378a0206c051 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:38.828265 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64336595-a9c5-4c94-836c-378a0206c051 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:38.841832 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64336595-a9c5-4c94-836c-378a0206c051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-64336595-a9c5-4c94-836c-378a0206c051 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:38.846829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:38.848342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 177/348] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:38.849244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64336595-a9c5-4c94-836c-378a0206c051 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:38.889356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:38.906444 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77706c-ccaf-4983-9548-85551e15e135 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:38.946959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d990a-8ce5-4b81-b930-d0a06704f63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:38.947911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d990a-8ce5-4b81-b930-d0a06704f63f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:38.948355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d990a-8ce5-4b81-b930-d0a06704f63f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:38.948771 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d990a-8ce5-4b81-b930-d0a06704f63f None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:38.949629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d990a-8ce5-4b81-b930-d0a06704f63f None admin] RBAC: Adding query filter params (name=member) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:38.950215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:38.951373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d990a-8ce5-4b81-b930-d0a06704f63f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:38.953041 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d990a-8ce5-4b81-b930-d0a06704f63f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:38.962550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 172/349] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:38.963637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d990a-8ce5-4b81-b930-d0a06704f63f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:38.970596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64336595-a9c5-4c94-836c-378a0206c051 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:39.009714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce84438-e628-470b-ba97-edf0667bf3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:39.010288 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce84438-e628-470b-ba97-edf0667bf3a0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:39.010709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce84438-e628-470b-ba97-edf0667bf3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:39.010994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce84438-e628-470b-ba97-edf0667bf3a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:39.012044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce84438-e628-470b-ba97-edf0667bf3a0 None admin] RBAC: Adding query filter params (role.id=2ca1bca713f745ea9b40dc20cb5fa2a1, user.id=d0442007c633432ea0a73ce8bfee726a, scope.system=all) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:39.012160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce84438-e628-470b-ba97-edf0667bf3a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:39.013555 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce84438-e628-470b-ba97-edf0667bf3a0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:39.026943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 178/350] 10.222.0.73 () {56 vars in 1308 bytes} [Thu Aug 4 21:21:38 2022] GET /identity/v3/role_assignments?user.id=d0442007c633432ea0a73ce8bfee726a&scope.system=all&role.id=2ca1bca713f745ea9b40dc20cb5fa2a1 => generated 510 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:39.027613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce84438-e628-470b-ba97-edf0667bf3a0 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:39.106981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d990a-8ce5-4b81-b930-d0a06704f63f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:39.155061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d48d79-842b-4e08-9649-02be7a9b6f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:39.156169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d48d79-842b-4e08-9649-02be7a9b6f42 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:39.156635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d48d79-842b-4e08-9649-02be7a9b6f42 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:39.157009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d48d79-842b-4e08-9649-02be7a9b6f42 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:39.157615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce84438-e628-470b-ba97-edf0667bf3a0 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:39.157980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d48d79-842b-4e08-9649-02be7a9b6f42 None admin] RBAC: Adding query filter params (name=member) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:39.158409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d48d79-842b-4e08-9649-02be7a9b6f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:39.162129 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d48d79-842b-4e08-9649-02be7a9b6f42 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:39.173516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 173/351] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:39.176503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79c032-81d9-435a-a5b3-ec5ba19a1299 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:39.176909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79c032-81d9-435a-a5b3-ec5ba19a1299 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:39.177704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79c032-81d9-435a-a5b3-ec5ba19a1299 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:39.184301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 174/352] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:39.186258 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79c032-81d9-435a-a5b3-ec5ba19a1299 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:39.204071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afafc9-cda1-4f7e-a4fc-048f6167c881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:39.205042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afafc9-cda1-4f7e-a4fc-048f6167c881 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:39.205338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afafc9-cda1-4f7e-a4fc-048f6167c881 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:39.205615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afafc9-cda1-4f7e-a4fc-048f6167c881 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:39.206685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afafc9-cda1-4f7e-a4fc-048f6167c881 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:39.207031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afafc9-cda1-4f7e-a4fc-048f6167c881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:39.209479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afafc9-cda1-4f7e-a4fc-048f6167c881 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:39.221220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 179/353] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:39.221795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:39.226555 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afafc9-cda1-4f7e-a4fc-048f6167c881 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:39.235626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:39.331395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79c032-81d9-435a-a5b3-ec5ba19a1299 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:39.366424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afafc9-cda1-4f7e-a4fc-048f6167c881 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:39.378996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3db956-d0b1-428e-a4c9-d4d356537459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:39.379776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3db956-d0b1-428e-a4c9-d4d356537459 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:39.380118 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3db956-d0b1-428e-a4c9-d4d356537459 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:39.380398 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3db956-d0b1-428e-a4c9-d4d356537459 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:39.408371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d14028-1367-4598-9674-7a8fe538feab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:39.408991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d14028-1367-4598-9674-7a8fe538feab None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:39.409270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d14028-1367-4598-9674-7a8fe538feab None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:39.409547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d14028-1367-4598-9674-7a8fe538feab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:39.459622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3db956-d0b1-428e-a4c9-d4d356537459 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:39.461093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3db956-d0b1-428e-a4c9-d4d356537459 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:39.476196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:39.497567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d14028-1367-4598-9674-7a8fe538feab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:39.499351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d14028-1367-4598-9674-7a8fe538feab None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:39.503298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:39.599357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 175/354] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:39.600538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8469b1-149f-49d9-80a4-45b653a3242f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:39.600825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8469b1-149f-49d9-80a4-45b653a3242f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:39.601098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8469b1-149f-49d9-80a4-45b653a3242f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:39.620943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bf8469b1-149f-49d9-80a4-45b653a3242f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:39.622373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bf8469b1-149f-49d9-80a4-45b653a3242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:39.633560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 180/355] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:39.634743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d14028-1367-4598-9674-7a8fe538feab None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:39.703441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 176/356] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:39.704530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a10d9-8bbc-4bc3-aca0-1cb3c33c183e None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:39.704878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a10d9-8bbc-4bc3-aca0-1cb3c33c183e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:39.705144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a10d9-8bbc-4bc3-aca0-1cb3c33c183e None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:39.706571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 177/357] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:39.707332 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a10d9-8bbc-4bc3-aca0-1cb3c33c183e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:39.757319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d14028-1367-4598-9674-7a8fe538feab None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:39.762417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:39.768187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:39.798873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d33c04-723d-4123-94c8-d80de9881277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:39.799504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d33c04-723d-4123-94c8-d80de9881277 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:39.800237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d33c04-723d-4123-94c8-d80de9881277 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:39.800397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d33c04-723d-4123-94c8-d80de9881277 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:39.830161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a10d9-8bbc-4bc3-aca0-1cb3c33c183e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:39.874826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5cf07-ffa7-4a0f-9253-3c863e5912db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:39.875484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5cf07-ffa7-4a0f-9253-3c863e5912db None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:39.875838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5cf07-ffa7-4a0f-9253-3c863e5912db None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:39.876185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5cf07-ffa7-4a0f-9253-3c863e5912db None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:39.877353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5cf07-ffa7-4a0f-9253-3c863e5912db None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:39.879411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5cf07-ffa7-4a0f-9253-3c863e5912db None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:39.881086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 178/358] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:39 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:39.881932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5cf07-ffa7-4a0f-9253-3c863e5912db None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:39.891095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d33c04-723d-4123-94c8-d80de9881277 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:39.892674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d33c04-723d-4123-94c8-d80de9881277 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:39.999501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5cf07-ffa7-4a0f-9253-3c863e5912db None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:40.009589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:40.015419 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 181/359] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:40.016667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5da76-6ec2-440e-a076-adb8098291db None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:40.016960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5da76-6ec2-440e-a076-adb8098291db None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:40.017236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5da76-6ec2-440e-a076-adb8098291db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:40.026315 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:40.037177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0db5da76-6ec2-440e-a076-adb8098291db None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:40.038562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0db5da76-6ec2-440e-a076-adb8098291db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:40.042636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430be092-fcb3-4157-8e63-0f577be6340c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:40.044754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430be092-fcb3-4157-8e63-0f577be6340c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:40.044754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430be092-fcb3-4157-8e63-0f577be6340c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:40.044754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430be092-fcb3-4157-8e63-0f577be6340c None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:40.044970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430be092-fcb3-4157-8e63-0f577be6340c None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:40.046436 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430be092-fcb3-4157-8e63-0f577be6340c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:40.048839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 179/360] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:39 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:40.050521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430be092-fcb3-4157-8e63-0f577be6340c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:40.120551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 182/361] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:40.121400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5da76-6ec2-440e-a076-adb8098291db None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:40.182139 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430be092-fcb3-4157-8e63-0f577be6340c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:40.229111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a952-355e-4c3b-9f6d-62ab139dd190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:40.230220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a952-355e-4c3b-9f6d-62ab139dd190 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:40.230707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a952-355e-4c3b-9f6d-62ab139dd190 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:40.231165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a952-355e-4c3b-9f6d-62ab139dd190 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:40.248686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6a952-355e-4c3b-9f6d-62ab139dd190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:40.252054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5da76-6ec2-440e-a076-adb8098291db None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-18e6a952-355e-4c3b-9f6d-62ab139dd190 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:40.254385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:40.256113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 180/362] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:40.257293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a952-355e-4c3b-9f6d-62ab139dd190 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:40.299334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:40.305572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09e2c9-5ef1-4ba1-84c0-b767abb3acde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:40.306133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09e2c9-5ef1-4ba1-84c0-b767abb3acde None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:40.306416 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09e2c9-5ef1-4ba1-84c0-b767abb3acde None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:40.306692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09e2c9-5ef1-4ba1-84c0-b767abb3acde None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:40.309230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:40.313387 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09e2c9-5ef1-4ba1-84c0-b767abb3acde None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:40.318177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09e2c9-5ef1-4ba1-84c0-b767abb3acde None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:40.318177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 183/363] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:40 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:40.318177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09e2c9-5ef1-4ba1-84c0-b767abb3acde None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:40.377293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a952-355e-4c3b-9f6d-62ab139dd190 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:40.422608 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4e36c-ec6f-43d7-a467-3c862fdb9d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:40.423498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4e36c-ec6f-43d7-a467-3c862fdb9d20 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:40.423924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4e36c-ec6f-43d7-a467-3c862fdb9d20 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:40.424342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4e36c-ec6f-43d7-a467-3c862fdb9d20 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:40.426285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4e36c-ec6f-43d7-a467-3c862fdb9d20 None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:40.427841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4e36c-ec6f-43d7-a467-3c862fdb9d20 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:40.429470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 181/364] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:40 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:40.431390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4e36c-ec6f-43d7-a467-3c862fdb9d20 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:40.449525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09e2c9-5ef1-4ba1-84c0-b767abb3acde None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:40.490782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d52542-8810-426f-9a42-80b264fc61bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:40.491440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d52542-8810-426f-9a42-80b264fc61bb None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:40.491725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d52542-8810-426f-9a42-80b264fc61bb None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:40.492005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d52542-8810-426f-9a42-80b264fc61bb None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:40.493160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d52542-8810-426f-9a42-80b264fc61bb None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:40.494691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d52542-8810-426f-9a42-80b264fc61bb None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:40.496276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 184/365] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:40 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:40.497118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d52542-8810-426f-9a42-80b264fc61bb None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:40.555200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4e36c-ec6f-43d7-a467-3c862fdb9d20 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:40.559353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:40.584538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:40.595431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c185c0f-41c3-444c-b693-173bfeb2c99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:40.596387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c185c0f-41c3-444c-b693-173bfeb2c99a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:40.596673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c185c0f-41c3-444c-b693-173bfeb2c99a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:40.596948 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c185c0f-41c3-444c-b693-173bfeb2c99a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:40.612269 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c185c0f-41c3-444c-b693-173bfeb2c99a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-3c185c0f-41c3-444c-b693-173bfeb2c99a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:40.616957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:40.617889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 182/366] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:21:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:40.619262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c185c0f-41c3-444c-b693-173bfeb2c99a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:40.628768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d52542-8810-426f-9a42-80b264fc61bb None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:40.671318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd418c0-dffd-4070-add9-661495e6a4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:40.671947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd418c0-dffd-4070-add9-661495e6a4b8 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:40.672227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd418c0-dffd-4070-add9-661495e6a4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:40.672501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd418c0-dffd-4070-add9-661495e6a4b8 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:40.673242 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd418c0-dffd-4070-add9-661495e6a4b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:40.673547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd418c0-dffd-4070-add9-661495e6a4b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:40.674936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd418c0-dffd-4070-add9-661495e6a4b8 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:40.683622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 185/367] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:40.684272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd418c0-dffd-4070-add9-661495e6a4b8 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:40.748050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c185c0f-41c3-444c-b693-173bfeb2c99a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:40.784413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f891c-3be0-42e9-bc18-af530e1ac4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:40.785701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f891c-3be0-42e9-bc18-af530e1ac4b7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:40.786065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f891c-3be0-42e9-bc18-af530e1ac4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:40.786458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f891c-3be0-42e9-bc18-af530e1ac4b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:40.787250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f891c-3be0-42e9-bc18-af530e1ac4b7 None admin] RBAC: Adding query filter params (group.id=c6bd13e6146d4f0a9449359150cdc609, role.id=36291c067d1949cfae0d704585f036c8, scope.project.id=6d126b37c8ae4111b880ecd5592237b8) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:40.787560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f891c-3be0-42e9-bc18-af530e1ac4b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:40.789002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f891c-3be0-42e9-bc18-af530e1ac4b7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:40.794301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 183/368] 10.222.0.73 () {56 vars in 1376 bytes} [Thu Aug 4 21:21:40 2022] GET /identity/v3/role_assignments?group.id=c6bd13e6146d4f0a9449359150cdc609&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=36291c067d1949cfae0d704585f036c8 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:40.795073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f891c-3be0-42e9-bc18-af530e1ac4b7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:40.810129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd418c0-dffd-4070-add9-661495e6a4b8 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:40.829432 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:40.841305 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:40.850391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab4478-2a87-4296-97e2-2395d15edd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:40.851066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab4478-2a87-4296-97e2-2395d15edd76 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:40.851431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab4478-2a87-4296-97e2-2395d15edd76 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:40.851680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab4478-2a87-4296-97e2-2395d15edd76 None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:40.852903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab4478-2a87-4296-97e2-2395d15edd76 None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:40.854393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab4478-2a87-4296-97e2-2395d15edd76 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:40.856250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 186/369] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:40 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:40.856810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab4478-2a87-4296-97e2-2395d15edd76 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:40.921409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f891c-3be0-42e9-bc18-af530e1ac4b7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:40.964436 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b1586-241f-4108-a625-9615ae671573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:40.965055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b1586-241f-4108-a625-9615ae671573 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:40.965333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b1586-241f-4108-a625-9615ae671573 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:40.965604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b1586-241f-4108-a625-9615ae671573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:40.966465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b1586-241f-4108-a625-9615ae671573 None admin] RBAC: Adding query filter params (group.id=c6bd13e6146d4f0a9449359150cdc609, role.id=36291c067d1949cfae0d704585f036c8, scope.project.id=d64e4c325db8448dadb785b8df946d6c) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:40.966775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b1586-241f-4108-a625-9615ae671573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:40.968189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b1586-241f-4108-a625-9615ae671573 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:40.974313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab4478-2a87-4296-97e2-2395d15edd76 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:40.982176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 184/370] 10.222.0.73 () {56 vars in 1376 bytes} [Thu Aug 4 21:21:40 2022] GET /identity/v3/role_assignments?group.id=c6bd13e6146d4f0a9449359150cdc609&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=36291c067d1949cfae0d704585f036c8 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:40.982176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b1586-241f-4108-a625-9615ae671573 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:41.024140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88f3da-0581-4a65-ab8e-fa693b6eeabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:41.024810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88f3da-0581-4a65-ab8e-fa693b6eeabb None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:41.025164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88f3da-0581-4a65-ab8e-fa693b6eeabb None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:41.025510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88f3da-0581-4a65-ab8e-fa693b6eeabb None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:41.026363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88f3da-0581-4a65-ab8e-fa693b6eeabb None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:41.026738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88f3da-0581-4a65-ab8e-fa693b6eeabb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:41.028220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88f3da-0581-4a65-ab8e-fa693b6eeabb None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:41.043217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 187/371] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:41.043217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88f3da-0581-4a65-ab8e-fa693b6eeabb None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:41.112133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:41.118360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:41.130580 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b1586-241f-4108-a625-9615ae671573 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:41.162861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88f3da-0581-4a65-ab8e-fa693b6eeabb None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:41.178700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2306bdd-5dda-44f4-9b2f-2835aa557db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:41.180216 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2306bdd-5dda-44f4-9b2f-2835aa557db8 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:41.180706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2306bdd-5dda-44f4-9b2f-2835aa557db8 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:41.181472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2306bdd-5dda-44f4-9b2f-2835aa557db8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:41.205211 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a518ed-55b6-4203-822a-72235ab1a4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:41.206158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a518ed-55b6-4203-822a-72235ab1a4e3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:41.206447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a518ed-55b6-4203-822a-72235ab1a4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:41.206781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a518ed-55b6-4203-822a-72235ab1a4e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:41.207570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a518ed-55b6-4203-822a-72235ab1a4e3 None admin] RBAC: Adding query filter params (group.id=c6bd13e6146d4f0a9449359150cdc609, role.id=677f038038444f229f95fc80c3166f24, scope.project.id=6d126b37c8ae4111b880ecd5592237b8) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:41.208061 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a518ed-55b6-4203-822a-72235ab1a4e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:41.210037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a518ed-55b6-4203-822a-72235ab1a4e3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:41.217047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 188/372] 10.222.0.73 () {56 vars in 1376 bytes} [Thu Aug 4 21:21:41 2022] GET /identity/v3/role_assignments?group.id=c6bd13e6146d4f0a9449359150cdc609&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=677f038038444f229f95fc80c3166f24 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:41.218284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a518ed-55b6-4203-822a-72235ab1a4e3 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:41.269073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2306bdd-5dda-44f4-9b2f-2835aa557db8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:41.271896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2306bdd-5dda-44f4-9b2f-2835aa557db8 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:41.345161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a518ed-55b6-4203-822a-72235ab1a4e3 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:41.372144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:41.373443 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:41.382604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 185/373] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:41.387301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9cd6d-f75c-4cad-b2db-b85db95d48a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:41.388297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9cd6d-f75c-4cad-b2db-b85db95d48a5 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:41.389019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9cd6d-f75c-4cad-b2db-b85db95d48a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:41.389474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2306bdd-5dda-44f4-9b2f-2835aa557db8 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:41.390052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9cd6d-f75c-4cad-b2db-b85db95d48a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:41.471107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9cd6d-f75c-4cad-b2db-b85db95d48a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:41.473567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9cd6d-f75c-4cad-b2db-b85db95d48a5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:41.505828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2306bdd-5dda-44f4-9b2f-2835aa557db8 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:41.537431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99e591-fe20-4b84-970f-1091f61db01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:41.538123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99e591-fe20-4b84-970f-1091f61db01f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:41.538457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99e591-fe20-4b84-970f-1091f61db01f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:41.538729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99e591-fe20-4b84-970f-1091f61db01f None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:41.540028 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99e591-fe20-4b84-970f-1091f61db01f None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:41.541550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99e591-fe20-4b84-970f-1091f61db01f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:41.543629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 186/374] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:41 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:41.549173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99e591-fe20-4b84-970f-1091f61db01f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:41.600225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 189/375] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:41.605475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9cd6d-f75c-4cad-b2db-b85db95d48a5 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:41.615101 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:41.633511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:41.666033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99e591-fe20-4b84-970f-1091f61db01f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:41.695578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ac66a-89bd-400a-a001-c84a81dd1093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:41.696232 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ac66a-89bd-400a-a001-c84a81dd1093 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:41.696512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ac66a-89bd-400a-a001-c84a81dd1093 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:41.696782 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ac66a-89bd-400a-a001-c84a81dd1093 None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:41.697967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ac66a-89bd-400a-a001-c84a81dd1093 None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:41.699562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ac66a-89bd-400a-a001-c84a81dd1093 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:41.701442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 187/376] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:41 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:41.707939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ac66a-89bd-400a-a001-c84a81dd1093 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:41.811659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ac66a-89bd-400a-a001-c84a81dd1093 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:41.849808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deba938-748a-4233-943b-41bfadfb9646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:41.850580 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deba938-748a-4233-943b-41bfadfb9646 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:41.850800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deba938-748a-4233-943b-41bfadfb9646 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:41.851073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deba938-748a-4233-943b-41bfadfb9646 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:41.851908 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deba938-748a-4233-943b-41bfadfb9646 None admin] RBAC: Adding query filter params (group.id=c6bd13e6146d4f0a9449359150cdc609, role.id=677f038038444f229f95fc80c3166f24, scope.project.id=d64e4c325db8448dadb785b8df946d6c) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:41.852222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deba938-748a-4233-943b-41bfadfb9646 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:41.853642 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deba938-748a-4233-943b-41bfadfb9646 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:41.858404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:41.860858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 188/377] 10.222.0.73 () {56 vars in 1376 bytes} [Thu Aug 4 21:21:41 2022] GET /identity/v3/role_assignments?group.id=c6bd13e6146d4f0a9449359150cdc609&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=677f038038444f229f95fc80c3166f24 => generated 260 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:41.881507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9cd6d-f75c-4cad-b2db-b85db95d48a5 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:41.951386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d970bc3-fd5d-46d5-9c6f-764121e649d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:41.952060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d970bc3-fd5d-46d5-9c6f-764121e649d5 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:41.952340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d970bc3-fd5d-46d5-9c6f-764121e649d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:41.952611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d970bc3-fd5d-46d5-9c6f-764121e649d5 None admin] PATH_INFO: `/v3/projects/1a7e2e1e30694bd4b3a5931f10dc9fa8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:41.953847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d970bc3-fd5d-46d5-9c6f-764121e649d5 None admin] RBAC: Authorizing `identity:get_project(project_id=1a7e2e1e30694bd4b3a5931f10dc9fa8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:41.955452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d970bc3-fd5d-46d5-9c6f-764121e649d5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:41.962324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 190/378] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:41 2022] GET /identity/v3/projects/1a7e2e1e30694bd4b3a5931f10dc9fa8 => generated 344 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:41.964175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deba938-748a-4233-943b-41bfadfb9646 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:42.096460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deba938-748a-4233-943b-41bfadfb9646 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:42.133340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c3e75-8644-4ee5-92aa-983dfac227f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:42.133970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c3e75-8644-4ee5-92aa-983dfac227f9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:42.134284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c3e75-8644-4ee5-92aa-983dfac227f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:42.134612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c3e75-8644-4ee5-92aa-983dfac227f9 None admin] PATH_INFO: `/v3/groups/4ebb601c7bda4e3bb796c702fe5d3c63` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:42.141632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c3e75-8644-4ee5-92aa-983dfac227f9 None admin] RBAC: Authorizing `identity:get_group(group_id=4ebb601c7bda4e3bb796c702fe5d3c63)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:42.143247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c3e75-8644-4ee5-92aa-983dfac227f9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:42.146040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 189/379] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:41 2022] GET /identity/v3/groups/4ebb601c7bda4e3bb796c702fe5d3c63 => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:42.151704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d970bc3-fd5d-46d5-9c6f-764121e649d5 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:42.292280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:42.303495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d970bc3-fd5d-46d5-9c6f-764121e649d5 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:42.348375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4589ef2-4ae4-460d-8230-c2b95bffc9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:42.348997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4589ef2-4ae4-460d-8230-c2b95bffc9cf None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:42.349280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4589ef2-4ae4-460d-8230-c2b95bffc9cf None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:42.349553 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4589ef2-4ae4-460d-8230-c2b95bffc9cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:42.350519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4589ef2-4ae4-460d-8230-c2b95bffc9cf None admin] RBAC: Adding query filter params (group.id=4ebb601c7bda4e3bb796c702fe5d3c63, role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=1a7e2e1e30694bd4b3a5931f10dc9fa8) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:42.351554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4589ef2-4ae4-460d-8230-c2b95bffc9cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:42.352260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4589ef2-4ae4-460d-8230-c2b95bffc9cf None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:42.362679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 191/380] 10.222.0.73 () {56 vars in 1376 bytes} [Thu Aug 4 21:21:42 2022] GET /identity/v3/role_assignments?group.id=4ebb601c7bda4e3bb796c702fe5d3c63&scope.project.id=1a7e2e1e30694bd4b3a5931f10dc9fa8&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:43.146613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099d0e6-a152-465a-9191-9e19a0379ca5 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:43.147154 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099d0e6-a152-465a-9191-9e19a0379ca5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:43.147231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099d0e6-a152-465a-9191-9e19a0379ca5 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:43.148838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 190/381] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:43.153825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480b6c8-cd55-4cd2-a323-a434dfbfdac1 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:43.154240 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480b6c8-cd55-4cd2-a323-a434dfbfdac1 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:43.154400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480b6c8-cd55-4cd2-a323-a434dfbfdac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:43.176067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0480b6c8-cd55-4cd2-a323-a434dfbfdac1 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:43.177275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0480b6c8-cd55-4cd2-a323-a434dfbfdac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:43.263642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 192/382] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:43.294817 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099d0e6-a152-465a-9191-9e19a0379ca5 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:43.359366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:43.418930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099d0e6-a152-465a-9191-9e19a0379ca5 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:43.427899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebafd24-2801-43c6-8f37-08fa95a5d49c None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:43.428213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebafd24-2801-43c6-8f37-08fa95a5d49c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:43.428652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebafd24-2801-43c6-8f37-08fa95a5d49c None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:43.430366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 193/383] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:43.436097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ec584-a160-48c9-b8a5-f6aab0598f69 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:43.436398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ec584-a160-48c9-b8a5-f6aab0598f69 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:43.436643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ec584-a160-48c9-b8a5-f6aab0598f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:43.455324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d91ec584-a160-48c9-b8a5-f6aab0598f69 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:43.456553 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d91ec584-a160-48c9-b8a5-f6aab0598f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:43.461719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea211a-4758-4411-9c37-b98bd525592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:43.462597 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea211a-4758-4411-9c37-b98bd525592b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:43.463045 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea211a-4758-4411-9c37-b98bd525592b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:43.463467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea211a-4758-4411-9c37-b98bd525592b None admin] PATH_INFO: `/v3/roles/member` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:43.476037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea211a-4758-4411-9c37-b98bd525592b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-a8ea211a-4758-4411-9c37-b98bd525592b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:43.482521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:43.483617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:43.484645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:43.487467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 191/384] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:43.493372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea211a-4758-4411-9c37-b98bd525592b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:43.574397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 194/385] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:43.597107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ec584-a160-48c9-b8a5-f6aab0598f69 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:43.614570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:43.687391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea211a-4758-4411-9c37-b98bd525592b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:43.706165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:43.721468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ec584-a160-48c9-b8a5-f6aab0598f69 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:43.725990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc488e-e559-458a-bee4-00b2428a8183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:43.726895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc488e-e559-458a-bee4-00b2428a8183 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:43.727377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc488e-e559-458a-bee4-00b2428a8183 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:43.727851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc488e-e559-458a-bee4-00b2428a8183 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:43.728749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc488e-e559-458a-bee4-00b2428a8183 None admin] RBAC: Adding query filter params (name=member) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:43.729261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc488e-e559-458a-bee4-00b2428a8183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:43.730883 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc488e-e559-458a-bee4-00b2428a8183 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:43.741952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 192/386] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:43.747559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc488e-e559-458a-bee4-00b2428a8183 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:43.793018 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2efc8f-8f14-4e7e-a116-9e08b3c7dd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:43.794083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2efc8f-8f14-4e7e-a116-9e08b3c7dd4a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:43.794559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2efc8f-8f14-4e7e-a116-9e08b3c7dd4a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:43.795025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2efc8f-8f14-4e7e-a116-9e08b3c7dd4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:43.822410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2efc8f-8f14-4e7e-a116-9e08b3c7dd4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-de2efc8f-8f14-4e7e-a116-9e08b3c7dd4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:43.829027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:43.829909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:43.830744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:43.842198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 195/387] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:43.856635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2efc8f-8f14-4e7e-a116-9e08b3c7dd4a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:43.921352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc488e-e559-458a-bee4-00b2428a8183 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:43.963467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:43.971056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb26072-1e01-4c0e-9800-5e225d07ff1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:43.972720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb26072-1e01-4c0e-9800-5e225d07ff1a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:43.973285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb26072-1e01-4c0e-9800-5e225d07ff1a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:43.973700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb26072-1e01-4c0e-9800-5e225d07ff1a None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:43.975136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb26072-1e01-4c0e-9800-5e225d07ff1a None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:43.977004 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb26072-1e01-4c0e-9800-5e225d07ff1a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:43.984082 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 193/388] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:43 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:43.984969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6513768-6d1c-416e-895c-2846e3c51b06 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:43.985542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6513768-6d1c-416e-895c-2846e3c51b06 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:43.985822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6513768-6d1c-416e-895c-2846e3c51b06 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:43.987450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 194/389] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:43.988243 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6513768-6d1c-416e-895c-2846e3c51b06 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:44.002253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2efc8f-8f14-4e7e-a116-9e08b3c7dd4a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:44.040486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749caef-3946-41d3-92f3-38373afabf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:44.041478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749caef-3946-41d3-92f3-38373afabf1b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.041823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749caef-3946-41d3-92f3-38373afabf1b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.042211 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749caef-3946-41d3-92f3-38373afabf1b None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.043021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749caef-3946-41d3-92f3-38373afabf1b None admin] RBAC: Adding query filter params (name=member) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:44.043398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749caef-3946-41d3-92f3-38373afabf1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:44.044852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749caef-3946-41d3-92f3-38373afabf1b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:44.054213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:44.054711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 196/390] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:44.056160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbad3c-60eb-4a42-a31a-0f85054ca6d9 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.056523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbad3c-60eb-4a42-a31a-0f85054ca6d9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.057034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbad3c-60eb-4a42-a31a-0f85054ca6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.076470 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-dacbad3c-60eb-4a42-a31a-0f85054ca6d9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:44.077771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-dacbad3c-60eb-4a42-a31a-0f85054ca6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:44.127675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6513768-6d1c-416e-895c-2846e3c51b06 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:44.165954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdb3ce-f1f7-4942-9cb7-bc573f4fa7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:44.166618 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdb3ce-f1f7-4942-9cb7-bc573f4fa7a4 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.167009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdb3ce-f1f7-4942-9cb7-bc573f4fa7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.167347 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdb3ce-f1f7-4942-9cb7-bc573f4fa7a4 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.168572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdb3ce-f1f7-4942-9cb7-bc573f4fa7a4 None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:44.170279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdb3ce-f1f7-4942-9cb7-bc573f4fa7a4 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:44.170727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 197/391] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:44.172125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbad3c-60eb-4a42-a31a-0f85054ca6d9 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:44.179786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 195/392] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:43 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:44.185750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdb3ce-f1f7-4942-9cb7-bc573f4fa7a4 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:44.256108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:44.292025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbad3c-60eb-4a42-a31a-0f85054ca6d9 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:44.331945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2dd95-a275-43b1-b94a-08a5f90f41b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:44.332966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2dd95-a275-43b1-b94a-08a5f90f41b4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.333238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2dd95-a275-43b1-b94a-08a5f90f41b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.333584 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2dd95-a275-43b1-b94a-08a5f90f41b4 None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.334885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2dd95-a275-43b1-b94a-08a5f90f41b4 None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:44.336463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2dd95-a275-43b1-b94a-08a5f90f41b4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:44.336815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdb3ce-f1f7-4942-9cb7-bc573f4fa7a4 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:44.338240 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 198/393] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:44 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:44.339098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2dd95-a275-43b1-b94a-08a5f90f41b4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:44.356459 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:44.387947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9b6ca-452b-4c1b-bfef-30fa0f419da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:44.388534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9b6ca-452b-4c1b-bfef-30fa0f419da3 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.388888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9b6ca-452b-4c1b-bfef-30fa0f419da3 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.389291 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9b6ca-452b-4c1b-bfef-30fa0f419da3 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8/groups/c6bd13e6146d4f0a9449359150cdc609/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.392010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9b6ca-452b-4c1b-bfef-30fa0f419da3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d126b37c8ae4111b880ecd5592237b8, group_id=c6bd13e6146d4f0a9449359150cdc609, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:44.393584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9b6ca-452b-4c1b-bfef-30fa0f419da3 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:44.453547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2dd95-a275-43b1-b94a-08a5f90f41b4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:44.474752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 196/394] 10.222.0.73 () {58 vars in 1442 bytes} [Thu Aug 4 21:21:44 2022] PUT /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8/groups/c6bd13e6146d4f0a9449359150cdc609/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:44.474752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9b6ca-452b-4c1b-bfef-30fa0f419da3 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:44.505832 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f5316-b41d-4c8e-b6e3-e293c0bc2b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:44.507160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f5316-b41d-4c8e-b6e3-e293c0bc2b4d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.507403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f5316-b41d-4c8e-b6e3-e293c0bc2b4d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.507674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f5316-b41d-4c8e-b6e3-e293c0bc2b4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.510772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:44.530032 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f5316-b41d-4c8e-b6e3-e293c0bc2b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-c81f5316-b41d-4c8e-b6e3-e293c0bc2b4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:44.533093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:44.534201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:44.536029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 199/395] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:44.540486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d13f4-0e43-4520-9de9-3afffb1509ca None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.540486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d13f4-0e43-4520-9de9-3afffb1509ca None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.540486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d13f4-0e43-4520-9de9-3afffb1509ca None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.542170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 200/396] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:44.543189 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d13f4-0e43-4520-9de9-3afffb1509ca None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:44.610989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9b6ca-452b-4c1b-bfef-30fa0f419da3 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:44.655470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ea7c9-7db2-4ce4-8b0e-ca966b9a9a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:44.656125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ea7c9-7db2-4ce4-8b0e-ca966b9a9a7f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.656407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ea7c9-7db2-4ce4-8b0e-ca966b9a9a7f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.657331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ea7c9-7db2-4ce4-8b0e-ca966b9a9a7f None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.658552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ea7c9-7db2-4ce4-8b0e-ca966b9a9a7f None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:44.660072 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ea7c9-7db2-4ce4-8b0e-ca966b9a9a7f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:44.662224 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 197/397] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:44 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:44.663211 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36e64-b520-44dd-b490-78c9642b7bd1 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.663495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36e64-b520-44dd-b490-78c9642b7bd1 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.663767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36e64-b520-44dd-b490-78c9642b7bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.684939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-11a36e64-b520-44dd-b490-78c9642b7bd1 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:44.686256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-11a36e64-b520-44dd-b490-78c9642b7bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:44.697199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d13f4-0e43-4520-9de9-3afffb1509ca None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:44.737993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:44.746926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daeeed-5b23-4ee9-90e9-fed054a3dd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:44.751282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daeeed-5b23-4ee9-90e9-fed054a3dd2c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.751282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daeeed-5b23-4ee9-90e9-fed054a3dd2c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.751282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daeeed-5b23-4ee9-90e9-fed054a3dd2c None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.751282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47daeeed-5b23-4ee9-90e9-fed054a3dd2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:44.751282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47daeeed-5b23-4ee9-90e9-fed054a3dd2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:44.754330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47daeeed-5b23-4ee9-90e9-fed054a3dd2c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:44.764417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 201/398] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:44.764993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daeeed-5b23-4ee9-90e9-fed054a3dd2c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:44.782081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 198/399] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:44.790536 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a36e64-b520-44dd-b490-78c9642b7bd1 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:44.856952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:44.913845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daeeed-5b23-4ee9-90e9-fed054a3dd2c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:44.928415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a36e64-b520-44dd-b490-78c9642b7bd1 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:44.952064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3757e-4acb-47bb-97ca-350a36df88ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:44.952779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3757e-4acb-47bb-97ca-350a36df88ea None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.953063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3757e-4acb-47bb-97ca-350a36df88ea None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.953337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3757e-4acb-47bb-97ca-350a36df88ea None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c/groups/c6bd13e6146d4f0a9449359150cdc609/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.955338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3757e-4acb-47bb-97ca-350a36df88ea None admin] RBAC: Authorizing `identity:create_grant(project_id=d64e4c325db8448dadb785b8df946d6c, group_id=c6bd13e6146d4f0a9449359150cdc609, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:44.957017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3757e-4acb-47bb-97ca-350a36df88ea None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:44.963780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462da363-eba9-40fa-bafd-d8236e14d986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:44.964429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462da363-eba9-40fa-bafd-d8236e14d986 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:44.964709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462da363-eba9-40fa-bafd-d8236e14d986 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:44.964981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462da363-eba9-40fa-bafd-d8236e14d986 None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:44.966171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462da363-eba9-40fa-bafd-d8236e14d986 None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:44.968682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462da363-eba9-40fa-bafd-d8236e14d986 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:44.970466 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 199/400] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:44 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:44.971086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462da363-eba9-40fa-bafd-d8236e14d986 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:45.014717 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:45.024487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 202/401] 10.222.0.73 () {58 vars in 1442 bytes} [Thu Aug 4 21:21:44 2022] PUT /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c/groups/c6bd13e6146d4f0a9449359150cdc609/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:45.027033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79e326-0d6d-4dae-ac39-e5b66e72a3cb None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:45.027033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79e326-0d6d-4dae-ac39-e5b66e72a3cb None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:45.027033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79e326-0d6d-4dae-ac39-e5b66e72a3cb None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:45.027639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 203/402] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:45.028441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79e326-0d6d-4dae-ac39-e5b66e72a3cb None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:45.107712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462da363-eba9-40fa-bafd-d8236e14d986 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:45.149122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50cfd9-02b8-4ebe-b8ea-9e27fcdc3614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:45.149785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50cfd9-02b8-4ebe-b8ea-9e27fcdc3614 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:45.150100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50cfd9-02b8-4ebe-b8ea-9e27fcdc3614 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:45.150380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50cfd9-02b8-4ebe-b8ea-9e27fcdc3614 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:45.164052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50cfd9-02b8-4ebe-b8ea-9e27fcdc3614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-8d50cfd9-02b8-4ebe-b8ea-9e27fcdc3614 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:45.170220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:45.171629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:45.172530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 200/403] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:45.173683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566e4b1-d8e1-40e6-bd04-38f26f9e15a9 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:45.173683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566e4b1-d8e1-40e6-bd04-38f26f9e15a9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:45.173904 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566e4b1-d8e1-40e6-bd04-38f26f9e15a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:45.177521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79e326-0d6d-4dae-ac39-e5b66e72a3cb None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:45.194590 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-3566e4b1-d8e1-40e6-bd04-38f26f9e15a9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:45.195810 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3566e4b1-d8e1-40e6-bd04-38f26f9e15a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:45.200388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:45.232949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c428de4-3e70-4c3f-9aec-6630957ec7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:45.233605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428de4-3e70-4c3f-9aec-6630957ec7e0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:45.234157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428de4-3e70-4c3f-9aec-6630957ec7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:45.234487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428de4-3e70-4c3f-9aec-6630957ec7e0 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:45.236827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c428de4-3e70-4c3f-9aec-6630957ec7e0 None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:45.238764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c428de4-3e70-4c3f-9aec-6630957ec7e0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:45.244123 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 204/404] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:45 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:45.244123 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c428de4-3e70-4c3f-9aec-6630957ec7e0 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:45.283671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 201/405] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:45.284714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566e4b1-d8e1-40e6-bd04-38f26f9e15a9 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:45.328572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:45.406304 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c428de4-3e70-4c3f-9aec-6630957ec7e0 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:45.406639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566e4b1-d8e1-40e6-bd04-38f26f9e15a9 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:45.447640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeee9e-2dfd-42d4-bdf8-3a358d9d5c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:45.448674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeee9e-2dfd-42d4-bdf8-3a358d9d5c81 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:45.449170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeee9e-2dfd-42d4-bdf8-3a358d9d5c81 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:45.449670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeee9e-2dfd-42d4-bdf8-3a358d9d5c81 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8/groups/c6bd13e6146d4f0a9449359150cdc609/roles/677f038038444f229f95fc80c3166f24` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:45.451779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afd9e6-bc33-420c-847e-b1e7a46b7a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:45.452397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afd9e6-bc33-420c-847e-b1e7a46b7a57 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:45.452659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeee9e-2dfd-42d4-bdf8-3a358d9d5c81 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d126b37c8ae4111b880ecd5592237b8, group_id=c6bd13e6146d4f0a9449359150cdc609, role_id=677f038038444f229f95fc80c3166f24)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:45.452768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afd9e6-bc33-420c-847e-b1e7a46b7a57 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:45.452929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afd9e6-bc33-420c-847e-b1e7a46b7a57 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:45.453635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afd9e6-bc33-420c-847e-b1e7a46b7a57 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:45.453937 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afd9e6-bc33-420c-847e-b1e7a46b7a57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:45.454972 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeee9e-2dfd-42d4-bdf8-3a358d9d5c81 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:45.455323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afd9e6-bc33-420c-847e-b1e7a46b7a57 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:45.465011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 205/406] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:45.465661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afd9e6-bc33-420c-847e-b1e7a46b7a57 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:45.527294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:45.559392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 202/407] 10.222.0.73 () {58 vars in 1442 bytes} [Thu Aug 4 21:21:45 2022] PUT /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8/groups/c6bd13e6146d4f0a9449359150cdc609/roles/677f038038444f229f95fc80c3166f24 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:45.561683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeee9e-2dfd-42d4-bdf8-3a358d9d5c81 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:45.600977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afd9e6-bc33-420c-847e-b1e7a46b7a57 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:45.685752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a8b54-8cb3-4a99-b441-d90e1e012a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:45.686837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a8b54-8cb3-4a99-b441-d90e1e012a9d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:45.687692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a8b54-8cb3-4a99-b441-d90e1e012a9d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:45.688809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a8b54-8cb3-4a99-b441-d90e1e012a9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:45.712816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a8b54-8cb3-4a99-b441-d90e1e012a9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-b32a8b54-8cb3-4a99-b441-d90e1e012a9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:45.719874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:45.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:45.721635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:45.726247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 206/408] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:21:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:45.732165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a8b54-8cb3-4a99-b441-d90e1e012a9d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:45.763494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:45.780779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeee9e-2dfd-42d4-bdf8-3a358d9d5c81 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:45.841480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccffae-2722-49a3-9870-37aa61a986a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:45.842543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccffae-2722-49a3-9870-37aa61a986a4 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:45.843035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccffae-2722-49a3-9870-37aa61a986a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:45.843512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccffae-2722-49a3-9870-37aa61a986a4 None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:45.844909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ccffae-2722-49a3-9870-37aa61a986a4 None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:45.847155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ccffae-2722-49a3-9870-37aa61a986a4 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:45.850377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 203/409] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:45 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:45.859142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccffae-2722-49a3-9870-37aa61a986a4 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:45.900028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a8b54-8cb3-4a99-b441-d90e1e012a9d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:45.914918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:45.977450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7bf7a-109f-40c8-9ca4-8930e9d34198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:45.978100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7bf7a-109f-40c8-9ca4-8930e9d34198 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:45.978386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7bf7a-109f-40c8-9ca4-8930e9d34198 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:45.978660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7bf7a-109f-40c8-9ca4-8930e9d34198 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:45.979383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7bf7a-109f-40c8-9ca4-8930e9d34198 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:45.979692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7bf7a-109f-40c8-9ca4-8930e9d34198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:45.981078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7bf7a-109f-40c8-9ca4-8930e9d34198 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:45.996724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 207/410] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:46.002941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7bf7a-109f-40c8-9ca4-8930e9d34198 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:46.065844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccffae-2722-49a3-9870-37aa61a986a4 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:46.140077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25733462-0c0e-4508-8064-d93750127feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:46.141503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25733462-0c0e-4508-8064-d93750127feb None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:46.142096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25733462-0c0e-4508-8064-d93750127feb None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:46.142578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25733462-0c0e-4508-8064-d93750127feb None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:46.145826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25733462-0c0e-4508-8064-d93750127feb None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:46.152729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25733462-0c0e-4508-8064-d93750127feb None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:46.155094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 204/411] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:45 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:46.163105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25733462-0c0e-4508-8064-d93750127feb None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:46.169242 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7bf7a-109f-40c8-9ca4-8930e9d34198 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:46.213009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd9ba0-b1f8-48ef-a923-ad497d74e21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:46.215671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd9ba0-b1f8-48ef-a923-ad497d74e21b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:46.216921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd9ba0-b1f8-48ef-a923-ad497d74e21b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:46.217945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd9ba0-b1f8-48ef-a923-ad497d74e21b None admin] PATH_INFO: `/v3/groups/4ebb601c7bda4e3bb796c702fe5d3c63` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:46.219079 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:46.222056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd9ba0-b1f8-48ef-a923-ad497d74e21b None admin] RBAC: Authorizing `identity:get_group(group_id=4ebb601c7bda4e3bb796c702fe5d3c63)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:46.225300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd9ba0-b1f8-48ef-a923-ad497d74e21b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:46.231525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 208/412] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:46 2022] GET /identity/v3/groups/4ebb601c7bda4e3bb796c702fe5d3c63 => generated 229 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:46.239235 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd9ba0-b1f8-48ef-a923-ad497d74e21b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:46.303645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:46.313617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25733462-0c0e-4508-8064-d93750127feb None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:46.352658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f66bd-acdd-4f0f-b9c7-1728eb8e6e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:46.353928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f66bd-acdd-4f0f-b9c7-1728eb8e6e6c None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:46.354437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f66bd-acdd-4f0f-b9c7-1728eb8e6e6c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:46.354904 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f66bd-acdd-4f0f-b9c7-1728eb8e6e6c None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c/groups/c6bd13e6146d4f0a9449359150cdc609/roles/677f038038444f229f95fc80c3166f24` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:46.357494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f66bd-acdd-4f0f-b9c7-1728eb8e6e6c None admin] RBAC: Authorizing `identity:create_grant(project_id=d64e4c325db8448dadb785b8df946d6c, group_id=c6bd13e6146d4f0a9449359150cdc609, role_id=677f038038444f229f95fc80c3166f24)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:46.359338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f66bd-acdd-4f0f-b9c7-1728eb8e6e6c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:46.402272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd9ba0-b1f8-48ef-a923-ad497d74e21b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:46.438877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d53f2-859b-45a4-bf17-1e05ea5ce200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:46.439841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d53f2-859b-45a4-bf17-1e05ea5ce200 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:46.440368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d53f2-859b-45a4-bf17-1e05ea5ce200 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:46.440859 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d53f2-859b-45a4-bf17-1e05ea5ce200 None admin] PATH_INFO: `/v3/projects/1a7e2e1e30694bd4b3a5931f10dc9fa8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:46.442941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d53f2-859b-45a4-bf17-1e05ea5ce200 None admin] RBAC: Authorizing `identity:get_project(project_id=1a7e2e1e30694bd4b3a5931f10dc9fa8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:46.444688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d53f2-859b-45a4-bf17-1e05ea5ce200 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:46.446937 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 209/413] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:46 2022] GET /identity/v3/projects/1a7e2e1e30694bd4b3a5931f10dc9fa8 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:46.454979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d53f2-859b-45a4-bf17-1e05ea5ce200 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:46.468081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 205/414] 10.222.0.73 () {58 vars in 1442 bytes} [Thu Aug 4 21:21:46 2022] PUT /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c/groups/c6bd13e6146d4f0a9449359150cdc609/roles/677f038038444f229f95fc80c3166f24 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:21:46.602899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d53f2-859b-45a4-bf17-1e05ea5ce200 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:46.606933 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:46.645400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2f452-8cd6-445d-a394-36cfaeff875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:46.646108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2f452-8cd6-445d-a394-36cfaeff875c None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:46.646396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2f452-8cd6-445d-a394-36cfaeff875c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:46.646670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2f452-8cd6-445d-a394-36cfaeff875c None admin] PATH_INFO: `/v3/projects/1a7e2e1e30694bd4b3a5931f10dc9fa8/groups/4ebb601c7bda4e3bb796c702fe5d3c63/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:46.648651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2f452-8cd6-445d-a394-36cfaeff875c None admin] RBAC: Authorizing `identity:create_grant(project_id=1a7e2e1e30694bd4b3a5931f10dc9fa8, group_id=4ebb601c7bda4e3bb796c702fe5d3c63, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:46.650177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2f452-8cd6-445d-a394-36cfaeff875c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:46.751628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 210/415] 10.222.0.73 () {58 vars in 1442 bytes} [Thu Aug 4 21:21:46 2022] PUT /identity/v3/projects/1a7e2e1e30694bd4b3a5931f10dc9fa8/groups/4ebb601c7bda4e3bb796c702fe5d3c63/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:21:46.760689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9db51-c056-41f6-a093-ff54c470f5b9 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:46.761022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9db51-c056-41f6-a093-ff54c470f5b9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:46.761256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9db51-c056-41f6-a093-ff54c470f5b9 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:46.762928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 206/416] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:46.768504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb53a5-d05a-46a9-987d-271cfa1be305 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:46.769795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb53a5-d05a-46a9-987d-271cfa1be305 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:46.770540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb53a5-d05a-46a9-987d-271cfa1be305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:46.794239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d2eb53a5-d05a-46a9-987d-271cfa1be305 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:46.795883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d2eb53a5-d05a-46a9-987d-271cfa1be305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:46.871593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 211/417] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:46.914963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9db51-c056-41f6-a093-ff54c470f5b9 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:47.023726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:47.043160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9db51-c056-41f6-a093-ff54c470f5b9 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:47.084672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9aa4e2-11b8-4350-a258-af26dbf9b908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:47.085325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9aa4e2-11b8-4350-a258-af26dbf9b908 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:47.085630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9aa4e2-11b8-4350-a258-af26dbf9b908 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:47.085912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9aa4e2-11b8-4350-a258-af26dbf9b908 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:47.096767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9aa4e2-11b8-4350-a258-af26dbf9b908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-1a9aa4e2-11b8-4350-a258-af26dbf9b908 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:47.102451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:47.103341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:47.104120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 207/418] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:47.113044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb53a5-d05a-46a9-987d-271cfa1be305 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:47.217753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f3978-0b2f-4f79-a16d-fe0401a97f90 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:47.218071 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f3978-0b2f-4f79-a16d-fe0401a97f90 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:47.218358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f3978-0b2f-4f79-a16d-fe0401a97f90 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:47.221029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 208/419] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:47.225186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa7d6d-8997-4c54-8592-585566c498bf None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:47.225476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa7d6d-8997-4c54-8592-585566c498bf None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:47.225758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa7d6d-8997-4c54-8592-585566c498bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:47.234496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:47.243058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb53a5-d05a-46a9-987d-271cfa1be305 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:47.263057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-30fa7d6d-8997-4c54-8592-585566c498bf None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:47.264596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-30fa7d6d-8997-4c54-8592-585566c498bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:47.282095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e996876-c9a2-4b25-91e8-fc830206520f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:47.282931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996876-c9a2-4b25-91e8-fc830206520f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:47.283363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996876-c9a2-4b25-91e8-fc830206520f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:47.283767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996876-c9a2-4b25-91e8-fc830206520f None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:47.284904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e996876-c9a2-4b25-91e8-fc830206520f None admin] RBAC: Adding query filter params (name=member) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:47.285356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e996876-c9a2-4b25-91e8-fc830206520f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:47.286919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e996876-c9a2-4b25-91e8-fc830206520f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:47.296123 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 212/420] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:47.303738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e996876-c9a2-4b25-91e8-fc830206520f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:47.412542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e996876-c9a2-4b25-91e8-fc830206520f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:47.448833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d211b81-e283-48b8-8363-6db37fff39d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:47.449838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d211b81-e283-48b8-8363-6db37fff39d1 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:47.450335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d211b81-e283-48b8-8363-6db37fff39d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:47.450888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d211b81-e283-48b8-8363-6db37fff39d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:47.451852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 209/421] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:47.477983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:47.495470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa7d6d-8997-4c54-8592-585566c498bf None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:47.529241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d211b81-e283-48b8-8363-6db37fff39d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:47.531030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d211b81-e283-48b8-8363-6db37fff39d1 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:47.640293 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 213/422] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:47.646977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d211b81-e283-48b8-8363-6db37fff39d1 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:47.685390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:47.729627 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa7d6d-8997-4c54-8592-585566c498bf None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:47.757955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:47.803177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf68cd-1c6b-4e2a-afae-40c159cbd61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:47.803831 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf68cd-1c6b-4e2a-afae-40c159cbd61b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:47.804119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf68cd-1c6b-4e2a-afae-40c159cbd61b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:47.804398 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf68cd-1c6b-4e2a-afae-40c159cbd61b None admin] PATH_INFO: `/v3/roles/member` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:47.826719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf68cd-1c6b-4e2a-afae-40c159cbd61b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-f1bf68cd-1c6b-4e2a-afae-40c159cbd61b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:47.833337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:47.834686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 21:21:47.835780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:47.839686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 210/423] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:47.840912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf68cd-1c6b-4e2a-afae-40c159cbd61b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:47.850040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d211b81-e283-48b8-8363-6db37fff39d1 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:47.884713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61d0e9-cc00-402e-85a2-2491a89613fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:47.885358 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61d0e9-cc00-402e-85a2-2491a89613fb None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:47.885644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61d0e9-cc00-402e-85a2-2491a89613fb None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:47.885920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61d0e9-cc00-402e-85a2-2491a89613fb None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:47.887129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61d0e9-cc00-402e-85a2-2491a89613fb None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:47.888567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61d0e9-cc00-402e-85a2-2491a89613fb None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:47.890597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 214/424] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:47 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:47.896284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61d0e9-cc00-402e-85a2-2491a89613fb None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:47.985810 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf68cd-1c6b-4e2a-afae-40c159cbd61b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:48.031529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae62bc9-29a5-4e57-962d-4e7569f9f1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:48.032462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae62bc9-29a5-4e57-962d-4e7569f9f1a4 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:48.033016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae62bc9-29a5-4e57-962d-4e7569f9f1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:48.033299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae62bc9-29a5-4e57-962d-4e7569f9f1a4 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:48.034182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae62bc9-29a5-4e57-962d-4e7569f9f1a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:48.034504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae62bc9-29a5-4e57-962d-4e7569f9f1a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:48.037230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae62bc9-29a5-4e57-962d-4e7569f9f1a4 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:48.039758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:48.049886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 211/425] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:21:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:48.051798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed7da-afda-49d6-82fd-16bafe602acf None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:48.052297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed7da-afda-49d6-82fd-16bafe602acf None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:48.052713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed7da-afda-49d6-82fd-16bafe602acf None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:48.054965 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 212/426] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:48.056722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feed7da-afda-49d6-82fd-16bafe602acf None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:48.079696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61d0e9-cc00-402e-85a2-2491a89613fb None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:48.114157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:48.122610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c5da2-be4a-49b5-aac2-a32f8b69e79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:48.123737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c5da2-be4a-49b5-aac2-a32f8b69e79f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:48.124129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c5da2-be4a-49b5-aac2-a32f8b69e79f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:48.124508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c5da2-be4a-49b5-aac2-a32f8b69e79f None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:48.126344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c5da2-be4a-49b5-aac2-a32f8b69e79f None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:48.127817 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c5da2-be4a-49b5-aac2-a32f8b69e79f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:48.134219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 215/427] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:47 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:48.134219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecf7d7-ed58-4bd6-9aa3-3b94bce9565c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:48.134219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecf7d7-ed58-4bd6-9aa3-3b94bce9565c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:48.134219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecf7d7-ed58-4bd6-9aa3-3b94bce9565c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:48.161364 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-aeecf7d7-ed58-4bd6-9aa3-3b94bce9565c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:48.163612 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-aeecf7d7-ed58-4bd6-9aa3-3b94bce9565c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:48.187455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feed7da-afda-49d6-82fd-16bafe602acf None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:48.225885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dedae6e-c2b9-4625-9ba4-a3af0e4ebc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:48.227268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedae6e-c2b9-4625-9ba4-a3af0e4ebc3f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:48.227738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedae6e-c2b9-4625-9ba4-a3af0e4ebc3f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:48.228170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedae6e-c2b9-4625-9ba4-a3af0e4ebc3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:48.254030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 216/428] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:48.254030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeecf7d7-ed58-4bd6-9aa3-3b94bce9565c None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:48.323647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:48.328230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dedae6e-c2b9-4625-9ba4-a3af0e4ebc3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:48.329971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dedae6e-c2b9-4625-9ba4-a3af0e4ebc3f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:48.372336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeecf7d7-ed58-4bd6-9aa3-3b94bce9565c None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:48.403517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:48.419617 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d128c-46a8-402a-8a33-2b2ca44afb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:48.420164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d128c-46a8-402a-8a33-2b2ca44afb2a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:48.420450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d128c-46a8-402a-8a33-2b2ca44afb2a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:48.420727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d128c-46a8-402a-8a33-2b2ca44afb2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:48.421519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d128c-46a8-402a-8a33-2b2ca44afb2a None admin] RBAC: Adding query filter params (group.id=c6bd13e6146d4f0a9449359150cdc609, role.id=36291c067d1949cfae0d704585f036c8, scope.project.id=6d126b37c8ae4111b880ecd5592237b8) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:48.421829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d128c-46a8-402a-8a33-2b2ca44afb2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:48.423900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d128c-46a8-402a-8a33-2b2ca44afb2a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:48.431165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 217/429] 10.222.0.73 () {56 vars in 1376 bytes} [Thu Aug 4 21:21:48 2022] GET /identity/v3/role_assignments?group.id=c6bd13e6146d4f0a9449359150cdc609&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=36291c067d1949cfae0d704585f036c8 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:48.432238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d128c-46a8-402a-8a33-2b2ca44afb2a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:48.445459 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 213/430] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:48.451875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dedae6e-c2b9-4625-9ba4-a3af0e4ebc3f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:48.566952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d128c-46a8-402a-8a33-2b2ca44afb2a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:48.571753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:48.576953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dedae6e-c2b9-4625-9ba4-a3af0e4ebc3f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:48.617913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281ac8f-9492-473e-8c10-ed44e35479a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:48.618854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281ac8f-9492-473e-8c10-ed44e35479a9 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:48.619971 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281ac8f-9492-473e-8c10-ed44e35479a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:48.621788 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281ac8f-9492-473e-8c10-ed44e35479a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:48.641662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9381449-1408-4f5e-89ad-28f7d18978ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:48.642637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9381449-1408-4f5e-89ad-28f7d18978ee None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:48.643107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9381449-1408-4f5e-89ad-28f7d18978ee None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:48.643540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9381449-1408-4f5e-89ad-28f7d18978ee None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:48.645241 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9381449-1408-4f5e-89ad-28f7d18978ee None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:48.646070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281ac8f-9492-473e-8c10-ed44e35479a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:48.651494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9381449-1408-4f5e-89ad-28f7d18978ee None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:48.653647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 214/431] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:48 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-0281ac8f-9492-473e-8c10-ed44e35479a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:48.656105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:48.657213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:48.658199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:48.660063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9381449-1408-4f5e-89ad-28f7d18978ee None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:48.661932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 218/432] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:48.671608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281ac8f-9492-473e-8c10-ed44e35479a9 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:48.733478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:48.797002 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281ac8f-9492-473e-8c10-ed44e35479a9 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:48.814548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9381449-1408-4f5e-89ad-28f7d18978ee None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:48.835866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50fc28-c6ab-437b-841e-ec3665c59f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:48.836783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50fc28-c6ab-437b-841e-ec3665c59f2e None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:48.837240 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50fc28-c6ab-437b-841e-ec3665c59f2e None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:48.837683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50fc28-c6ab-437b-841e-ec3665c59f2e None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:48.838663 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50fc28-c6ab-437b-841e-ec3665c59f2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:48.839172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50fc28-c6ab-437b-841e-ec3665c59f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:48.840786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50fc28-c6ab-437b-841e-ec3665c59f2e None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:48.850143 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 219/433] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:48.856004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50fc28-c6ab-437b-841e-ec3665c59f2e None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:48.864955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f7c8d-7482-4c2d-b7fc-347223742fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:48.865921 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f7c8d-7482-4c2d-b7fc-347223742fb4 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:48.866437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f7c8d-7482-4c2d-b7fc-347223742fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:48.866917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f7c8d-7482-4c2d-b7fc-347223742fb4 None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:48.868293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f7c8d-7482-4c2d-b7fc-347223742fb4 None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:48.870038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f7c8d-7482-4c2d-b7fc-347223742fb4 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:48.872195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 215/434] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:48 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:48.877668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f7c8d-7482-4c2d-b7fc-347223742fb4 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:48.922045 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:48.988766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50fc28-c6ab-437b-841e-ec3665c59f2e None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:49.004921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:49.018582 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f7c8d-7482-4c2d-b7fc-347223742fb4 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:49.037433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8268e5-2b7a-45a4-98d5-3e723635ef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:49.038091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8268e5-2b7a-45a4-98d5-3e723635ef8f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.038380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8268e5-2b7a-45a4-98d5-3e723635ef8f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.038658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8268e5-2b7a-45a4-98d5-3e723635ef8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.066773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde52786-c956-4db6-b263-d4815cd5af6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:49.067538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde52786-c956-4db6-b263-d4815cd5af6c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.067882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde52786-c956-4db6-b263-d4815cd5af6c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.068166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde52786-c956-4db6-b263-d4815cd5af6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.068969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde52786-c956-4db6-b263-d4815cd5af6c None admin] RBAC: Adding query filter params (group.id=c6bd13e6146d4f0a9449359150cdc609, role.id=36291c067d1949cfae0d704585f036c8, scope.project.id=d64e4c325db8448dadb785b8df946d6c) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:49.069285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde52786-c956-4db6-b263-d4815cd5af6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:49.070761 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde52786-c956-4db6-b263-d4815cd5af6c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:49.079046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 216/435] 10.222.0.73 () {56 vars in 1376 bytes} [Thu Aug 4 21:21:48 2022] GET /identity/v3/role_assignments?group.id=c6bd13e6146d4f0a9449359150cdc609&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=36291c067d1949cfae0d704585f036c8 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:49.080528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e66be-bf26-4ab5-a3ba-bcea9580c85e None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.080820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e66be-bf26-4ab5-a3ba-bcea9580c85e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.081152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e66be-bf26-4ab5-a3ba-bcea9580c85e None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.086951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 217/436] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:49.088915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a36ab-b9a2-4ba4-9ba0-c80cac9ba193 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.089214 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a36ab-b9a2-4ba4-9ba0-c80cac9ba193 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.089497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a36ab-b9a2-4ba4-9ba0-c80cac9ba193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.116031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-3b7a36ab-b9a2-4ba4-9ba0-c80cac9ba193 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:49.117621 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3b7a36ab-b9a2-4ba4-9ba0-c80cac9ba193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:49.137375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8268e5-2b7a-45a4-98d5-3e723635ef8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:49.140795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8268e5-2b7a-45a4-98d5-3e723635ef8f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:49.217405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 218/437] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:49.244004 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a36ab-b9a2-4ba4-9ba0-c80cac9ba193 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:49.285762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 220/438] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:49.291494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8268e5-2b7a-45a4-98d5-3e723635ef8f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:49.292230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:49.303108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:49.379804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a36ab-b9a2-4ba4-9ba0-c80cac9ba193 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:49.398598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8268e5-2b7a-45a4-98d5-3e723635ef8f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:49.414795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ac5e7-269b-4df3-b1c5-318e31ce9442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:49.415429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ac5e7-269b-4df3-b1c5-318e31ce9442 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.415712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ac5e7-269b-4df3-b1c5-318e31ce9442 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.415983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ac5e7-269b-4df3-b1c5-318e31ce9442 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.427301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ac5e7-269b-4df3-b1c5-318e31ce9442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-4b6ac5e7-269b-4df3-b1c5-318e31ce9442 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:49.432231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:49.433095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:49.433892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:49.434534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f01b2f-0e8a-4534-8b15-b605bec521f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:49.434534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01b2f-0e8a-4534-8b15-b605bec521f5 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.434534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01b2f-0e8a-4534-8b15-b605bec521f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.434819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 219/439] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:49.434887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01b2f-0e8a-4534-8b15-b605bec521f5 None admin] PATH_INFO: `/v3/projects/6d126b37c8ae4111b880ecd5592237b8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.434951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4d0e-3945-42fa-a2c8-298a845e7c18 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.434951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4d0e-3945-42fa-a2c8-298a845e7c18 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.435667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4d0e-3945-42fa-a2c8-298a845e7c18 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.435888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f01b2f-0e8a-4534-8b15-b605bec521f5 None admin] RBAC: Authorizing `identity:get_project(project_id=6d126b37c8ae4111b880ecd5592237b8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:49.436590 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 220/440] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:49.437400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f01b2f-0e8a-4534-8b15-b605bec521f5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:49.439215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d4d0e-3945-42fa-a2c8-298a845e7c18 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:49.444239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 221/441] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:49 2022] GET /identity/v3/projects/6d126b37c8ae4111b880ecd5592237b8 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:49.444239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda59dc-e93a-4f4c-948d-437f5321d8e2 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.444239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda59dc-e93a-4f4c-948d-437f5321d8e2 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.444239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda59dc-e93a-4f4c-948d-437f5321d8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.460406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-2dda59dc-e93a-4f4c-948d-437f5321d8e2 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:49.461636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2dda59dc-e93a-4f4c-948d-437f5321d8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:49.530487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 222/442] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:49.531346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda59dc-e93a-4f4c-948d-437f5321d8e2 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:49.536706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:49.539254 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:49.551320 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d4d0e-3945-42fa-a2c8-298a845e7c18 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:49.586493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb055bd-50e1-4b0e-82f8-a448d2a9784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:49.587106 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb055bd-50e1-4b0e-82f8-a448d2a9784d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.587386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb055bd-50e1-4b0e-82f8-a448d2a9784d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.587661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb055bd-50e1-4b0e-82f8-a448d2a9784d None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.588365 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb055bd-50e1-4b0e-82f8-a448d2a9784d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:49.588670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb055bd-50e1-4b0e-82f8-a448d2a9784d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:49.590063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb055bd-50e1-4b0e-82f8-a448d2a9784d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:49.598000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 221/443] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:49.599918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb055bd-50e1-4b0e-82f8-a448d2a9784d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:49.628397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda59dc-e93a-4f4c-948d-437f5321d8e2 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:49.661862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043407ba-3847-4ddc-9d49-3ebe3e99c972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:49.662529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043407ba-3847-4ddc-9d49-3ebe3e99c972 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.662809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043407ba-3847-4ddc-9d49-3ebe3e99c972 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.663233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043407ba-3847-4ddc-9d49-3ebe3e99c972 None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.664389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043407ba-3847-4ddc-9d49-3ebe3e99c972 None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:49.665815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043407ba-3847-4ddc-9d49-3ebe3e99c972 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:49.667702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 223/444] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:49 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:49.668242 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043407ba-3847-4ddc-9d49-3ebe3e99c972 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:49.699643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb055bd-50e1-4b0e-82f8-a448d2a9784d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:49.733873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c703d75-375b-4924-a7d9-6517a884cbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:49.734533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c703d75-375b-4924-a7d9-6517a884cbdb None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.734816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c703d75-375b-4924-a7d9-6517a884cbdb None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.735086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c703d75-375b-4924-a7d9-6517a884cbdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.741641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:49.745402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c703d75-375b-4924-a7d9-6517a884cbdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-0c703d75-375b-4924-a7d9-6517a884cbdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:49.750766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:49.758539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 222/445] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:21:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:49.759340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c703d75-375b-4924-a7d9-6517a884cbdb None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:49.765130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:49.767036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043407ba-3847-4ddc-9d49-3ebe3e99c972 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:49.803628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5eb06-8cb1-4dfa-baba-ac026b1c4929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:49.804173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5eb06-8cb1-4dfa-baba-ac026b1c4929 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.804458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5eb06-8cb1-4dfa-baba-ac026b1c4929 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.804734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5eb06-8cb1-4dfa-baba-ac026b1c4929 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.851430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c703d75-375b-4924-a7d9-6517a884cbdb None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:49.872772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5eb06-8cb1-4dfa-baba-ac026b1c4929 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:49.874270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5eb06-8cb1-4dfa-baba-ac026b1c4929 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:49.884533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36821456-ec64-4200-9228-64f306e38236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:49.885141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821456-ec64-4200-9228-64f306e38236 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:49.885421 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821456-ec64-4200-9228-64f306e38236 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:49.885691 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821456-ec64-4200-9228-64f306e38236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:49.886493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36821456-ec64-4200-9228-64f306e38236 None admin] RBAC: Adding query filter params (group.id=c6bd13e6146d4f0a9449359150cdc609, role.id=677f038038444f229f95fc80c3166f24, scope.project.id=6d126b37c8ae4111b880ecd5592237b8) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:49.886799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36821456-ec64-4200-9228-64f306e38236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:49.888216 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36821456-ec64-4200-9228-64f306e38236 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:49.893797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 223/446] 10.222.0.73 () {56 vars in 1376 bytes} [Thu Aug 4 21:21:49 2022] GET /identity/v3/role_assignments?group.id=c6bd13e6146d4f0a9449359150cdc609&scope.project.id=6d126b37c8ae4111b880ecd5592237b8&role.id=677f038038444f229f95fc80c3166f24 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:49.895380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36821456-ec64-4200-9228-64f306e38236 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:49.963245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:49.984480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:49.992141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 224/447] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:49.997812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5eb06-8cb1-4dfa-baba-ac026b1c4929 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:50.012049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36821456-ec64-4200-9228-64f306e38236 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:50.053111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a391bd7-360e-4933-86cb-2b39e85c9a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:50.053802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a391bd7-360e-4933-86cb-2b39e85c9a4a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:50.054050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a391bd7-360e-4933-86cb-2b39e85c9a4a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:50.054317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a391bd7-360e-4933-86cb-2b39e85c9a4a None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:50.055029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a391bd7-360e-4933-86cb-2b39e85c9a4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:50.055334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a391bd7-360e-4933-86cb-2b39e85c9a4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:50.057147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a391bd7-360e-4933-86cb-2b39e85c9a4a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:50.067514 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 224/448] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:50.073268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a391bd7-360e-4933-86cb-2b39e85c9a4a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:50.110742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5eb06-8cb1-4dfa-baba-ac026b1c4929 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:50.154038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c990b-81a6-4f13-9f20-2f08bc2e6a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:50.154839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c990b-81a6-4f13-9f20-2f08bc2e6a81 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:50.155186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c990b-81a6-4f13-9f20-2f08bc2e6a81 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:50.155467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c990b-81a6-4f13-9f20-2f08bc2e6a81 None admin] PATH_INFO: `/v3/projects/d64e4c325db8448dadb785b8df946d6c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:50.156798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c990b-81a6-4f13-9f20-2f08bc2e6a81 None admin] RBAC: Authorizing `identity:get_project(project_id=d64e4c325db8448dadb785b8df946d6c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:50.158365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c990b-81a6-4f13-9f20-2f08bc2e6a81 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:50.160096 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 225/449] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:49 2022] GET /identity/v3/projects/d64e4c325db8448dadb785b8df946d6c => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:50.165896 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c990b-81a6-4f13-9f20-2f08bc2e6a81 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:50.193024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a391bd7-360e-4933-86cb-2b39e85c9a4a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:50.218681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:50.241304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931035c-5851-4f95-93d5-7f49e95165e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:50.242050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931035c-5851-4f95-93d5-7f49e95165e9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:50.242293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931035c-5851-4f95-93d5-7f49e95165e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:50.242532 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931035c-5851-4f95-93d5-7f49e95165e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:50.247005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:50.291817 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c990b-81a6-4f13-9f20-2f08bc2e6a81 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:50.328576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931035c-5851-4f95-93d5-7f49e95165e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:50.329949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931035c-5851-4f95-93d5-7f49e95165e9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:50.333966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f313c-2192-4c4f-917b-262d7870c13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:50.334643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f313c-2192-4c4f-917b-262d7870c13d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:50.334951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f313c-2192-4c4f-917b-262d7870c13d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:50.335202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f313c-2192-4c4f-917b-262d7870c13d None admin] PATH_INFO: `/v3/groups/c6bd13e6146d4f0a9449359150cdc609` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:50.336384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f313c-2192-4c4f-917b-262d7870c13d None admin] RBAC: Authorizing `identity:get_group(group_id=c6bd13e6146d4f0a9449359150cdc609)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:50.337807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f313c-2192-4c4f-917b-262d7870c13d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:50.339671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 226/450] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:50 2022] GET /identity/v3/groups/c6bd13e6146d4f0a9449359150cdc609 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:50.345362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f313c-2192-4c4f-917b-262d7870c13d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:50.444910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 225/451] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 21:21:50.450752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931035c-5851-4f95-93d5-7f49e95165e9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:50.464167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f313c-2192-4c4f-917b-262d7870c13d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:50.475468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:50.484138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:50.498905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c240a67-5359-4291-89d8-148474bac380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:50.499593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c240a67-5359-4291-89d8-148474bac380 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:50.500041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c240a67-5359-4291-89d8-148474bac380 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:50.500471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c240a67-5359-4291-89d8-148474bac380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:50.501562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c240a67-5359-4291-89d8-148474bac380 None admin] RBAC: Adding query filter params (group.id=c6bd13e6146d4f0a9449359150cdc609, role.id=677f038038444f229f95fc80c3166f24, scope.project.id=d64e4c325db8448dadb785b8df946d6c) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:50.501877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c240a67-5359-4291-89d8-148474bac380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:50.503434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c240a67-5359-4291-89d8-148474bac380 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:50.509508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 227/452] 10.222.0.73 () {56 vars in 1376 bytes} [Thu Aug 4 21:21:50 2022] GET /identity/v3/role_assignments?group.id=c6bd13e6146d4f0a9449359150cdc609&scope.project.id=d64e4c325db8448dadb785b8df946d6c&role.id=677f038038444f229f95fc80c3166f24 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:50.553375 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931035c-5851-4f95-93d5-7f49e95165e9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:50.588728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4b762-ce36-410f-9fc9-70c5aa4192a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:50.589296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4b762-ce36-410f-9fc9-70c5aa4192a0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:50.589577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4b762-ce36-410f-9fc9-70c5aa4192a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:50.589847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4b762-ce36-410f-9fc9-70c5aa4192a0 None admin] PATH_INFO: `/v3/projects/1a7e2e1e30694bd4b3a5931f10dc9fa8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:50.591104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4b762-ce36-410f-9fc9-70c5aa4192a0 None admin] RBAC: Authorizing `identity:get_project(project_id=1a7e2e1e30694bd4b3a5931f10dc9fa8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:50.592591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4b762-ce36-410f-9fc9-70c5aa4192a0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:50.594320 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 226/453] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:21:50 2022] GET /identity/v3/projects/1a7e2e1e30694bd4b3a5931f10dc9fa8 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:21:50.600787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c240a67-5359-4291-89d8-148474bac380 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:50.704385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c240a67-5359-4291-89d8-148474bac380 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:50.739580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e8446-4739-44b5-87de-0f7ee893526d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:50.740224 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e8446-4739-44b5-87de-0f7ee893526d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:50.740504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e8446-4739-44b5-87de-0f7ee893526d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:50.740777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e8446-4739-44b5-87de-0f7ee893526d None admin] PATH_INFO: `/v3/groups/4ebb601c7bda4e3bb796c702fe5d3c63` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:50.741934 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e8446-4739-44b5-87de-0f7ee893526d None admin] RBAC: Authorizing `identity:get_group(group_id=4ebb601c7bda4e3bb796c702fe5d3c63)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:50.743405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e8446-4739-44b5-87de-0f7ee893526d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:50.744974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 228/454] 10.222.0.73 () {56 vars in 1180 bytes} [Thu Aug 4 21:21:50 2022] GET /identity/v3/groups/4ebb601c7bda4e3bb796c702fe5d3c63 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:50.750860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4b762-ce36-410f-9fc9-70c5aa4192a0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:50.860720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4b762-ce36-410f-9fc9-70c5aa4192a0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:50.863123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:50.892774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3aee9-c52b-4ce4-b860-cff4b40ecd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:50.893596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3aee9-c52b-4ce4-b860-cff4b40ecd5b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:50.894601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3aee9-c52b-4ce4-b860-cff4b40ecd5b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:50.895183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3aee9-c52b-4ce4-b860-cff4b40ecd5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:50.895963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3aee9-c52b-4ce4-b860-cff4b40ecd5b None admin] RBAC: Adding query filter params (group.id=4ebb601c7bda4e3bb796c702fe5d3c63, role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=1a7e2e1e30694bd4b3a5931f10dc9fa8) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:50.896278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3aee9-c52b-4ce4-b860-cff4b40ecd5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:50.897701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3aee9-c52b-4ce4-b860-cff4b40ecd5b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:50.903471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 227/455] 10.222.0.73 () {56 vars in 1376 bytes} [Thu Aug 4 21:21:50 2022] GET /identity/v3/role_assignments?group.id=4ebb601c7bda4e3bb796c702fe5d3c63&scope.project.id=1a7e2e1e30694bd4b3a5931f10dc9fa8&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:53.377870 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6ffb5-d449-4c4e-a2a1-e134aa0be680 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:53.379145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6ffb5-d449-4c4e-a2a1-e134aa0be680 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:53.379640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6ffb5-d449-4c4e-a2a1-e134aa0be680 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:53.381433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 229/456] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:53.386833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa2a0-333f-44c7-870f-9a83628e5550 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:53.387636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa2a0-333f-44c7-870f-9a83628e5550 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:53.388115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa2a0-333f-44c7-870f-9a83628e5550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:53.415297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bfefa2a0-333f-44c7-870f-9a83628e5550 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:53.416827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bfefa2a0-333f-44c7-870f-9a83628e5550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:53.507414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 228/457] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:53.546656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6ffb5-d449-4c4e-a2a1-e134aa0be680 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:53.609451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:53.625849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae91ee-0d5b-4d58-bf7b-7a2ceed7c644 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:53.636715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae91ee-0d5b-4d58-bf7b-7a2ceed7c644 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:53.636715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae91ee-0d5b-4d58-bf7b-7a2ceed7c644 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:53.636715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 229/458] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:53.636715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f6ed3-499f-4c38-afdf-60ab1f9c2045 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:53.636715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f6ed3-499f-4c38-afdf-60ab1f9c2045 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:53.636715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f6ed3-499f-4c38-afdf-60ab1f9c2045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:53.667787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bc1f6ed3-499f-4c38-afdf-60ab1f9c2045 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:53.669209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bc1f6ed3-499f-4c38-afdf-60ab1f9c2045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:53.682689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6ffb5-d449-4c4e-a2a1-e134aa0be680 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:53.741355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 230/459] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:53.763901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cae42-b8eb-446f-9d60-4d3440663088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:53.764933 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cae42-b8eb-446f-9d60-4d3440663088 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:53.765071 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f6ed3-499f-4c38-afdf-60ab1f9c2045 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:53.765628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cae42-b8eb-446f-9d60-4d3440663088 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:53.766240 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cae42-b8eb-446f-9d60-4d3440663088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:53.780445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cae42-b8eb-446f-9d60-4d3440663088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:53.782343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cae42-b8eb-446f-9d60-4d3440663088 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:53.796241 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-525cae42-b8eb-446f-9d60-4d3440663088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:21:53.800983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:53.811168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:53.812028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:21:53.812028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:21:53.812028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:53.812028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:53.812028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 230/460] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:53.812028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd89916-e06b-402f-82c3-c0bb09d80fc8 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:53.812028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd89916-e06b-402f-82c3-c0bb09d80fc8 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:53.812028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd89916-e06b-402f-82c3-c0bb09d80fc8 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:53.814110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 231/461] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:53.815244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd89916-e06b-402f-82c3-c0bb09d80fc8 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:53.875668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f6ed3-499f-4c38-afdf-60ab1f9c2045 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:53.915161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6457171-68fd-42a1-8599-4c1a2b3d8848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:53.915161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6457171-68fd-42a1-8599-4c1a2b3d8848 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:53.915161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6457171-68fd-42a1-8599-4c1a2b3d8848 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:53.915161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6457171-68fd-42a1-8599-4c1a2b3d8848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:53.916117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:53.923937 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd89916-e06b-402f-82c3-c0bb09d80fc8 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:53.927454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6457171-68fd-42a1-8599-4c1a2b3d8848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:53.929169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6457171-68fd-42a1-8599-4c1a2b3d8848 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-f6457171-68fd-42a1-8599-4c1a2b3d8848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:21:53.946503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:53.952879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:53.953740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:21:53.953740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:21:53.953740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:53.953740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:53.953740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 231/462] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:53.954142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6993-76c1-42a6-a2b8-c84204befb14 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:53.954749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6993-76c1-42a6-a2b8-c84204befb14 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:53.955097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6993-76c1-42a6-a2b8-c84204befb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:53.963499 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823e5dc-8114-4568-936d-3cb704fce228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:53.964589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823e5dc-8114-4568-936d-3cb704fce228 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:53.965203 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823e5dc-8114-4568-936d-3cb704fce228 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:53.965668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823e5dc-8114-4568-936d-3cb704fce228 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:53.966672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823e5dc-8114-4568-936d-3cb704fce228 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:53.967189 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823e5dc-8114-4568-936d-3cb704fce228 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:53.968886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823e5dc-8114-4568-936d-3cb704fce228 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:53.976190 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-549d6993-76c1-42a6-a2b8-c84204befb14 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:53.977110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-549d6993-76c1-42a6-a2b8-c84204befb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:53.981425 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 232/463] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:53.982573 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823e5dc-8114-4568-936d-3cb704fce228 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:54.056724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 232/464] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:54.058463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d6993-76c1-42a6-a2b8-c84204befb14 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:54.070996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:54.089556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823e5dc-8114-4568-936d-3cb704fce228 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:54.125472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fd6bf-ea37-4831-8350-5208fb059b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:54.126464 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fd6bf-ea37-4831-8350-5208fb059b4a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:54.127006 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fd6bf-ea37-4831-8350-5208fb059b4a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:54.127476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fd6bf-ea37-4831-8350-5208fb059b4a None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:54.128983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fd6bf-ea37-4831-8350-5208fb059b4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:54.129486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fd6bf-ea37-4831-8350-5208fb059b4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:54.131063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fd6bf-ea37-4831-8350-5208fb059b4a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:54.143440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 233/465] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:54.144594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fd6bf-ea37-4831-8350-5208fb059b4a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:54.161084 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:54.176113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d6993-76c1-42a6-a2b8-c84204befb14 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:54.207544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be12f16-b975-47d8-b74f-dcc3faac0f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:54.208171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be12f16-b975-47d8-b74f-dcc3faac0f91 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:54.208451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be12f16-b975-47d8-b74f-dcc3faac0f91 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:54.208725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be12f16-b975-47d8-b74f-dcc3faac0f91 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:54.209504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be12f16-b975-47d8-b74f-dcc3faac0f91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:54.211083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be12f16-b975-47d8-b74f-dcc3faac0f91 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:54.249918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fd6bf-ea37-4831-8350-5208fb059b4a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:54.283447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ad481-54f6-43eb-9b22-55b8ebf18532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:54.284083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad481-54f6-43eb-9b22-55b8ebf18532 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:54.284369 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad481-54f6-43eb-9b22-55b8ebf18532 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:54.285325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad481-54f6-43eb-9b22-55b8ebf18532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:54.295664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ad481-54f6-43eb-9b22-55b8ebf18532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:54.297108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ad481-54f6-43eb-9b22-55b8ebf18532 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-bb5ad481-54f6-43eb-9b22-55b8ebf18532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:21:54.312913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:54.313989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:54.314947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:21:54.314947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:21:54.314947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:54.314947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:54.314947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 234/466] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:54.315215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ad481-54f6-43eb-9b22-55b8ebf18532 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:54.322355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 233/467] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:21:54 2022] POST /identity/v3/users => generated 246 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:54.335711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9f4ca-bbcc-46fe-8ba3-519735ca6de0 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:54.336005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9f4ca-bbcc-46fe-8ba3-519735ca6de0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:54.336280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9f4ca-bbcc-46fe-8ba3-519735ca6de0 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:54.337654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 234/468] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:54.338374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9f4ca-bbcc-46fe-8ba3-519735ca6de0 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:54.412393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:54.452146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:54.458544 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ad481-54f6-43eb-9b22-55b8ebf18532 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:54.504265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9f4ca-bbcc-46fe-8ba3-519735ca6de0 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:54.505838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d6243-d18a-4879-8ea0-4b6faf4102ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:54.506656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d6243-d18a-4879-8ea0-4b6faf4102ac None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:54.507114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d6243-d18a-4879-8ea0-4b6faf4102ac None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:54.507552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d6243-d18a-4879-8ea0-4b6faf4102ac None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:54.508312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d6243-d18a-4879-8ea0-4b6faf4102ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:54.509786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d6243-d18a-4879-8ea0-4b6faf4102ac None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:54.549560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0017fd8-f23a-4311-b8df-8c7e5967e3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:54.550699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0017fd8-f23a-4311-b8df-8c7e5967e3e2 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:54.551090 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0017fd8-f23a-4311-b8df-8c7e5967e3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:54.551373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0017fd8-f23a-4311-b8df-8c7e5967e3e2 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:54.552076 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0017fd8-f23a-4311-b8df-8c7e5967e3e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:54.552389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0017fd8-f23a-4311-b8df-8c7e5967e3e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:54.553865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0017fd8-f23a-4311-b8df-8c7e5967e3e2 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:54.574552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 235/469] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:54.575760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19684eb-8685-4005-855d-69d9659f1de4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:54.575760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19684eb-8685-4005-855d-69d9659f1de4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:54.575975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19684eb-8685-4005-855d-69d9659f1de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:54.594483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 235/470] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:21:54 2022] POST /identity/v3/users => generated 248 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:54.595157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-b19684eb-8685-4005-855d-69d9659f1de4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:54.595386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d6243-d18a-4879-8ea0-4b6faf4102ac None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:54.596504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b19684eb-8685-4005-855d-69d9659f1de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:54.678703 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 236/471] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:54.712490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19684eb-8685-4005-855d-69d9659f1de4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:54.756255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d6243-d18a-4879-8ea0-4b6faf4102ac None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:54.799101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:54.812575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582dc353-c929-4ce5-a728-140762f7e736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:54.813205 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582dc353-c929-4ce5-a728-140762f7e736 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:54.813484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582dc353-c929-4ce5-a728-140762f7e736 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:54.813758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582dc353-c929-4ce5-a728-140762f7e736 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:54.814494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582dc353-c929-4ce5-a728-140762f7e736 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:54.816057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582dc353-c929-4ce5-a728-140762f7e736 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:54.822293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:54.871016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19684eb-8685-4005-855d-69d9659f1de4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:54.908107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25dcb25-d6b7-45fe-accd-9b833a846dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:54.908759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25dcb25-d6b7-45fe-accd-9b833a846dcd None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:54.909037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25dcb25-d6b7-45fe-accd-9b833a846dcd None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:54.909307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25dcb25-d6b7-45fe-accd-9b833a846dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:54.919540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25dcb25-d6b7-45fe-accd-9b833a846dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:54.921073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25dcb25-d6b7-45fe-accd-9b833a846dcd None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:54.933540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 236/472] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:21:54 2022] POST /identity/v3/users => generated 248 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:54.935045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf90880-c192-4117-ba03-a9efefeb272b None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:54.935332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf90880-c192-4117-ba03-a9efefeb272b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:54.941027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf90880-c192-4117-ba03-a9efefeb272b None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-d25dcb25-d6b7-45fe-accd-9b833a846dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:21:54.941184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:54.942185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:54.943049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:21:54.943049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:21:54.943049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:54.943049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:54.943260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 237/473] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:54.943260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca381-e03d-4a29-89c2-bdbc8e54e6b0 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:54.943869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca381-e03d-4a29-89c2-bdbc8e54e6b0 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:54.943869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca381-e03d-4a29-89c2-bdbc8e54e6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:54.946146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 237/474] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:54.947127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25dcb25-d6b7-45fe-accd-9b833a846dcd None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:54.966951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ffbca381-e03d-4a29-89c2-bdbc8e54e6b0 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:54.968107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ffbca381-e03d-4a29-89c2-bdbc8e54e6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:55.064455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 238/475] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:55.073955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25dcb25-d6b7-45fe-accd-9b833a846dcd None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:55.084474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:55.105207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbca381-e03d-4a29-89c2-bdbc8e54e6b0 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:55.113506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291279b6-8cbe-4c1d-9d0d-2535d35d4b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:55.114184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291279b6-8cbe-4c1d-9d0d-2535d35d4b7e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:55.114473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291279b6-8cbe-4c1d-9d0d-2535d35d4b7e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:55.114746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291279b6-8cbe-4c1d-9d0d-2535d35d4b7e None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:55.115617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291279b6-8cbe-4c1d-9d0d-2535d35d4b7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:55.115995 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291279b6-8cbe-4c1d-9d0d-2535d35d4b7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:55.117422 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291279b6-8cbe-4c1d-9d0d-2535d35d4b7e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:55.131839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 238/476] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:55.143007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291279b6-8cbe-4c1d-9d0d-2535d35d4b7e None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:55.258405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbca381-e03d-4a29-89c2-bdbc8e54e6b0 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:55.268569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:55.277784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291279b6-8cbe-4c1d-9d0d-2535d35d4b7e None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:55.299041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a2532-1754-4803-bc2b-5472c0477a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:55.299720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a2532-1754-4803-bc2b-5472c0477a88 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:55.300005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a2532-1754-4803-bc2b-5472c0477a88 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:55.300642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a2532-1754-4803-bc2b-5472c0477a88 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:55.301806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a2532-1754-4803-bc2b-5472c0477a88 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:55.303228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a2532-1754-4803-bc2b-5472c0477a88 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:55.317223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299419cd-6160-4754-97e4-54c3ff836237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:55.321562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299419cd-6160-4754-97e4-54c3ff836237 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:55.321562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299419cd-6160-4754-97e4-54c3ff836237 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:55.321562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299419cd-6160-4754-97e4-54c3ff836237 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:55.321562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299419cd-6160-4754-97e4-54c3ff836237 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:55.321562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299419cd-6160-4754-97e4-54c3ff836237 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:55.340421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.sql.core [None req-a71a2532-1754-4803-bc2b-5472c0477a88 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 21:21:55.340421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 21:21:55.340421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [parameters: {'id': '93688adb3bc34f61b3e24f8ace00c88e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 21:21:55.340421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91912) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-a71a2532-1754-4803-bc2b-5472c0477a88 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application result.read() Aug 04 21:21:55.357565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return ref.to_dict() Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:21:55.358793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application yield resource Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise self.value Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application session.commit() Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 21:21:55.360047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self.session.flush() Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application compat.raise_( Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 21:21:55.361421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:21:55.363405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application result.read() Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:21:55.364717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [parameters: {'id': '93688adb3bc34f61b3e24f8ace00c88e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:55.365802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 21:21:55.366845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:55.368640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 239/477] 10.222.0.73 () {60 vars in 1131 bytes} [Thu Aug 4 21:21:55 2022] POST /identity/v3/roles => generated 144 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:55.372378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a2532-1754-4803-bc2b-5472c0477a88 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:55.419303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 239/478] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:21:55 2022] POST /identity/v3/users => generated 249 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:21:55.494572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a2532-1754-4803-bc2b-5472c0477a88 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:55.559967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d4be0-37d0-412d-b322-8a6b38cdf34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:55.560645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d4be0-37d0-412d-b322-8a6b38cdf34a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:55.560930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d4be0-37d0-412d-b322-8a6b38cdf34a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:55.561272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d4be0-37d0-412d-b322-8a6b38cdf34a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:55.577293 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d4be0-37d0-412d-b322-8a6b38cdf34a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-933d4be0-37d0-412d-b322-8a6b38cdf34a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:55.583109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:55.584066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:55.584893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 240/479] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:21:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:55.590380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299419cd-6160-4754-97e4-54c3ff836237 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:55.631070 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:55.715334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299419cd-6160-4754-97e4-54c3ff836237 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:55.754550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e431133-b99c-44f3-8815-7062ee31d47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:55.755284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e431133-b99c-44f3-8815-7062ee31d47f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:55.755564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e431133-b99c-44f3-8815-7062ee31d47f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:55.755839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e431133-b99c-44f3-8815-7062ee31d47f None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:55.756553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e431133-b99c-44f3-8815-7062ee31d47f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:55.756856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e431133-b99c-44f3-8815-7062ee31d47f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:55.758260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e431133-b99c-44f3-8815-7062ee31d47f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:55.771648 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 240/480] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:56.743530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314f66-0219-4c9a-bca9-011b18b6fbbb None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:56.744076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314f66-0219-4c9a-bca9-011b18b6fbbb None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:56.744076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314f66-0219-4c9a-bca9-011b18b6fbbb None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:56.745723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 241/481] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:56.750563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618afcb-a4c6-49ca-af69-1e79401d0125 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:56.750965 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618afcb-a4c6-49ca-af69-1e79401d0125 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:56.751132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618afcb-a4c6-49ca-af69-1e79401d0125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:56.779050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1618afcb-a4c6-49ca-af69-1e79401d0125 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:56.780174 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1618afcb-a4c6-49ca-af69-1e79401d0125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:56.868482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 241/482] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:56.909117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4314f66-0219-4c9a-bca9-011b18b6fbbb None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:56.950591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:57.042503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4314f66-0219-4c9a-bca9-011b18b6fbbb None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:57.089674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74578f62-03d3-44b3-9a87-cbbd8ce74fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:57.090351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74578f62-03d3-44b3-9a87-cbbd8ce74fb3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:57.090635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74578f62-03d3-44b3-9a87-cbbd8ce74fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:57.090909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74578f62-03d3-44b3-9a87-cbbd8ce74fb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:57.106905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74578f62-03d3-44b3-9a87-cbbd8ce74fb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-74578f62-03d3-44b3-9a87-cbbd8ce74fb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:57.112139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:57.113734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:57.115158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 242/483] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:21:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:57.121634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618afcb-a4c6-49ca-af69-1e79401d0125 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:57.150437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:57.241063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618afcb-a4c6-49ca-af69-1e79401d0125 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:57.289537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6f907-bfe9-43f1-8e98-da8f41dd862d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:57.290571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6f907-bfe9-43f1-8e98-da8f41dd862d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:57.291048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6f907-bfe9-43f1-8e98-da8f41dd862d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:57.291538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6f907-bfe9-43f1-8e98-da8f41dd862d None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:57.292441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6f907-bfe9-43f1-8e98-da8f41dd862d None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:57.292988 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6f907-bfe9-43f1-8e98-da8f41dd862d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:57.294584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6f907-bfe9-43f1-8e98-da8f41dd862d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:57.303955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 242/484] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:21:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:57.309041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74578f62-03d3-44b3-9a87-cbbd8ce74fb3 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:57.326936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50188c6d-f46c-4ae5-8c14-39ae2d2cda63 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:57.327565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50188c6d-f46c-4ae5-8c14-39ae2d2cda63 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:57.328055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50188c6d-f46c-4ae5-8c14-39ae2d2cda63 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:57.330507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 243/485] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:57.336669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250dceb-ea87-466c-a52a-48347cdb27e2 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:57.337317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250dceb-ea87-466c-a52a-48347cdb27e2 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:57.337783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250dceb-ea87-466c-a52a-48347cdb27e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:57.375243 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-4250dceb-ea87-466c-a52a-48347cdb27e2 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:57.376938 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4250dceb-ea87-466c-a52a-48347cdb27e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:57.426424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:57.431615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74578f62-03d3-44b3-9a87-cbbd8ce74fb3 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:57.468667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201b842-4dc1-480e-8c6e-880795616e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:57.469361 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201b842-4dc1-480e-8c6e-880795616e0a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:57.469647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201b842-4dc1-480e-8c6e-880795616e0a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:57.469920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201b842-4dc1-480e-8c6e-880795616e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:57.482432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201b842-4dc1-480e-8c6e-880795616e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:57.484418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201b842-4dc1-480e-8c6e-880795616e0a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:57.499844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 244/486] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-8201b842-4dc1-480e-8c6e-880795616e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:21:57.506897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:57.507736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:57.509149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:21:57.509149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:21:57.509149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:57.509149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:57.509149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 243/487] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:57.513661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250dceb-ea87-466c-a52a-48347cdb27e2 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:57.536789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:57.538901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201b842-4dc1-480e-8c6e-880795616e0a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:57.647651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201b842-4dc1-480e-8c6e-880795616e0a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:57.683152 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633c1cf-4f95-4b64-a125-2312d0a2fd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:57.683968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633c1cf-4f95-4b64-a125-2312d0a2fd09 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:57.684357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633c1cf-4f95-4b64-a125-2312d0a2fd09 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:57.684739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633c1cf-4f95-4b64-a125-2312d0a2fd09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:57.695678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633c1cf-4f95-4b64-a125-2312d0a2fd09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-d633c1cf-4f95-4b64-a125-2312d0a2fd09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:57.701332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:57.702267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:57.703195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 244/488] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:21:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:21:57.711695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633c1cf-4f95-4b64-a125-2312d0a2fd09 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:57.722503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:57.731785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250dceb-ea87-466c-a52a-48347cdb27e2 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:57.789414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e1527-51b3-4b56-8d73-4afe56783192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:57.790064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e1527-51b3-4b56-8d73-4afe56783192 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:57.790330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e1527-51b3-4b56-8d73-4afe56783192 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:57.790614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e1527-51b3-4b56-8d73-4afe56783192 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:57.791310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e1527-51b3-4b56-8d73-4afe56783192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:57.793788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e1527-51b3-4b56-8d73-4afe56783192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:57.793788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e1527-51b3-4b56-8d73-4afe56783192 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:57.814039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 245/489] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:57.814039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b60d-1eef-4871-9d4d-0266c1617297 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:57.814039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b60d-1eef-4871-9d4d-0266c1617297 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:57.814039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b60d-1eef-4871-9d4d-0266c1617297 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:57.814039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 246/490] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:57.814484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91b60d-1eef-4871-9d4d-0266c1617297 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:57.879077 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633c1cf-4f95-4b64-a125-2312d0a2fd09 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:57.897826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:57.919039 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7a4f6-38d1-4bed-91ea-a3f8b98d5df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:57.919921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7a4f6-38d1-4bed-91ea-a3f8b98d5df3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:57.920268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7a4f6-38d1-4bed-91ea-a3f8b98d5df3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:57.920649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7a4f6-38d1-4bed-91ea-a3f8b98d5df3 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:57.921599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7a4f6-38d1-4bed-91ea-a3f8b98d5df3 None admin] RBAC: Adding query filter params (name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:57.922030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7a4f6-38d1-4bed-91ea-a3f8b98d5df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:57.923644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7a4f6-38d1-4bed-91ea-a3f8b98d5df3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:57.933054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 245/491] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:21:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:57.934433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2adcc2-5d92-470a-be49-248778515659 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:57.941448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2adcc2-5d92-470a-be49-248778515659 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:57.941448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2adcc2-5d92-470a-be49-248778515659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:57.948243 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91b60d-1eef-4871-9d4d-0266c1617297 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:57.957111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-3c2adcc2-5d92-470a-be49-248778515659 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:57.958767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3c2adcc2-5d92-470a-be49-248778515659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:57.989150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb750ab1-53c1-4515-86f3-93b042345d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:57.989847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750ab1-53c1-4515-86f3-93b042345d8d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:57.990236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750ab1-53c1-4515-86f3-93b042345d8d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:57.990517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750ab1-53c1-4515-86f3-93b042345d8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:57.999332 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb750ab1-53c1-4515-86f3-93b042345d8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:58.000762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb750ab1-53c1-4515-86f3-93b042345d8d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-eb750ab1-53c1-4515-86f3-93b042345d8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:21:58.016030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:58.017007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:58.017920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:58.017920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:58.017920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:21:58.017920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:21:58.017920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:21:58.017920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:21:58.017920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:21:58.017920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:21:58.017920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:21:58.017920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:58.022058 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 247/492] 10.222.0.73 () {56 vars in 1093 bytes} [Thu Aug 4 21:21:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:58.022959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb750ab1-53c1-4515-86f3-93b042345d8d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:58.041645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:58.053722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 246/493] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:58.057848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2adcc2-5d92-470a-be49-248778515659 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:58.170967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2adcc2-5d92-470a-be49-248778515659 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:58.174867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb750ab1-53c1-4515-86f3-93b042345d8d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:58.201876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:58.208047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dd06d-cfb2-47ec-8a12-881c0cde6338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:58.208664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dd06d-cfb2-47ec-8a12-881c0cde6338 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.208949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dd06d-cfb2-47ec-8a12-881c0cde6338 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.209270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dd06d-cfb2-47ec-8a12-881c0cde6338 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.210048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dd06d-cfb2-47ec-8a12-881c0cde6338 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:58.210352 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dd06d-cfb2-47ec-8a12-881c0cde6338 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:58.210589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830164ac-0095-4dee-8ec5-ae98dcbe1e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:58.211228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830164ac-0095-4dee-8ec5-ae98dcbe1e3c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.211513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830164ac-0095-4dee-8ec5-ae98dcbe1e3c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.211813 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830164ac-0095-4dee-8ec5-ae98dcbe1e3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.211902 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dd06d-cfb2-47ec-8a12-881c0cde6338 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:58.222383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830164ac-0095-4dee-8ec5-ae98dcbe1e3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:58.230302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830164ac-0095-4dee-8ec5-ae98dcbe1e3c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-830164ac-0095-4dee-8ec5-ae98dcbe1e3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:21:58.238886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:58.239745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:58.240611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:21:58.240611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:21:58.240611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:58.240611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:58.244790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 248/494] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:58.245437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830164ac-0095-4dee-8ec5-ae98dcbe1e3c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:58.250704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 247/495] 10.222.0.73 () {56 vars in 1133 bytes} [Thu Aug 4 21:21:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:58.255804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c6f34-b94c-405a-8829-0cb6945efe3c None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.255804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c6f34-b94c-405a-8829-0cb6945efe3c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.256331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c6f34-b94c-405a-8829-0cb6945efe3c None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.257638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 248/496] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:58.258542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c6f34-b94c-405a-8829-0cb6945efe3c None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:58.318860 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:58.354901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830164ac-0095-4dee-8ec5-ae98dcbe1e3c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:58.371896 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c6f34-b94c-405a-8829-0cb6945efe3c None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:58.392428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac08a66-05b7-4e67-9c24-106a53e5d4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:58.393063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac08a66-05b7-4e67-9c24-106a53e5d4c2 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.393797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac08a66-05b7-4e67-9c24-106a53e5d4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.393797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac08a66-05b7-4e67-9c24-106a53e5d4c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.404947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac08a66-05b7-4e67-9c24-106a53e5d4c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:58.406988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4d6e2-9bd1-4f25-ab80-f85f221f5a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:58.407602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4d6e2-9bd1-4f25-ab80-f85f221f5a99 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.407882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4d6e2-9bd1-4f25-ab80-f85f221f5a99 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.408155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4d6e2-9bd1-4f25-ab80-f85f221f5a99 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.408843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4d6e2-9bd1-4f25-ab80-f85f221f5a99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:58.409148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4d6e2-9bd1-4f25-ab80-f85f221f5a99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:58.410578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4d6e2-9bd1-4f25-ab80-f85f221f5a99 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-3ac08a66-05b7-4e67-9c24-106a53e5d4c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:58.410677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:58.411994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 249/497] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:21:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:58.413682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac08a66-05b7-4e67-9c24-106a53e5d4c2 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:58.421830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 249/498] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:58.423005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd79c-f4d7-4e99-b031-c73c5cd79b3c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.423289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd79c-f4d7-4e99-b031-c73c5cd79b3c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.423608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd79c-f4d7-4e99-b031-c73c5cd79b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.442520 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-e87cd79c-f4d7-4e99-b031-c73c5cd79b3c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:58.443704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e87cd79c-f4d7-4e99-b031-c73c5cd79b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:58.459376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:58.511728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 250/499] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:58.512994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a911f32-6ba4-4cb7-b385-5f4f9b22823f None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.513289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a911f32-6ba4-4cb7-b385-5f4f9b22823f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.513948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a911f32-6ba4-4cb7-b385-5f4f9b22823f None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.515831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 251/500] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:21:58.516327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a911f32-6ba4-4cb7-b385-5f4f9b22823f None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:58.534963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac08a66-05b7-4e67-9c24-106a53e5d4c2 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:58.568092 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:58.569639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807b441-63cb-4c1c-80d7-045a3cfc8b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:58.570274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b441-63cb-4c1c-80d7-045a3cfc8b51 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.570552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b441-63cb-4c1c-80d7-045a3cfc8b51 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.570826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b441-63cb-4c1c-80d7-045a3cfc8b51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.623303 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a911f32-6ba4-4cb7-b385-5f4f9b22823f None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:58.636556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807b441-63cb-4c1c-80d7-045a3cfc8b51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:58.638400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807b441-63cb-4c1c-80d7-045a3cfc8b51 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:58.657464 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c76cd1-2080-4fbd-9d83-0a7ca016b29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:58.658106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c76cd1-2080-4fbd-9d83-0a7ca016b29d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.658392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c76cd1-2080-4fbd-9d83-0a7ca016b29d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.658668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c76cd1-2080-4fbd-9d83-0a7ca016b29d None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.659423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c76cd1-2080-4fbd-9d83-0a7ca016b29d None admin] RBAC: Adding query filter params (name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:58.659897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c76cd1-2080-4fbd-9d83-0a7ca016b29d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:58.661480 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c76cd1-2080-4fbd-9d83-0a7ca016b29d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:58.669441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 252/501] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:21:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:58.670254 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c76cd1-2080-4fbd-9d83-0a7ca016b29d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:58.679080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:58.738591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 250/502] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:58.739940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c5a5a-497d-4c31-abb6-c0ebb38dca10 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.740244 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c5a5a-497d-4c31-abb6-c0ebb38dca10 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.740510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c5a5a-497d-4c31-abb6-c0ebb38dca10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.757747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-fd2c5a5a-497d-4c31-abb6-c0ebb38dca10 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:21:58.758962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-fd2c5a5a-497d-4c31-abb6-c0ebb38dca10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:21:58.771804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c76cd1-2080-4fbd-9d83-0a7ca016b29d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:58.778496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:58.807776 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fff1b4-3bab-421c-adca-ccd121916cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:58.808425 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fff1b4-3bab-421c-adca-ccd121916cf6 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.808712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fff1b4-3bab-421c-adca-ccd121916cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.808998 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fff1b4-3bab-421c-adca-ccd121916cf6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.816931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fff1b4-3bab-421c-adca-ccd121916cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:58.818449 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fff1b4-3bab-421c-adca-ccd121916cf6 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:58.825212 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 251/503] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:21:58.825870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c5a5a-497d-4c31-abb6-c0ebb38dca10 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-09fff1b4-3bab-421c-adca-ccd121916cf6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:21:58.831111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:58.832069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:58.833040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:58.833040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:58.833040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:21:58.833040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:21:58.833040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:21:58.833040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:21:58.833040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:21:58.833040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:21:58.833040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:21:58.833040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:58.833040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 253/504] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:58.833779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fff1b4-3bab-421c-adca-ccd121916cf6 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:58.894791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:58.924954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c5a5a-497d-4c31-abb6-c0ebb38dca10 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:58.943512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fff1b4-3bab-421c-adca-ccd121916cf6 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:58.959409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fa6ce-1657-42da-8a14-338e57370d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:58.960095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fa6ce-1657-42da-8a14-338e57370d9d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.960383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fa6ce-1657-42da-8a14-338e57370d9d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.960656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fa6ce-1657-42da-8a14-338e57370d9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.970552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fa6ce-1657-42da-8a14-338e57370d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-bc1fa6ce-1657-42da-8a14-338e57370d9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:21:58.977213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:58.978299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:21:58.979141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:58.979778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb204171-88f8-44b8-8f5c-7a527aaf2d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:58.979778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb204171-88f8-44b8-8f5c-7a527aaf2d86 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:58.979918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 252/505] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:21:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:58.979992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb204171-88f8-44b8-8f5c-7a527aaf2d86 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:58.979992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb204171-88f8-44b8-8f5c-7a527aaf2d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:58.980111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fa6ce-1657-42da-8a14-338e57370d9d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:58.993847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb204171-88f8-44b8-8f5c-7a527aaf2d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:58.995320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb204171-88f8-44b8-8f5c-7a527aaf2d86 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-cb204171-88f8-44b8-8f5c-7a527aaf2d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:21:59.010625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:59.011665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:59.012735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:21:59.012735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:21:59.012735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:59.012735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:59.012735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 254/506] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:59.012999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb204171-88f8-44b8-8f5c-7a527aaf2d86 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:59.031970 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:59.084437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fa6ce-1657-42da-8a14-338e57370d9d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:59.113258 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:59.113877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb204171-88f8-44b8-8f5c-7a527aaf2d86 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:59.119130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5b570-90ea-4118-866c-65287f10e481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:59.119775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5b570-90ea-4118-866c-65287f10e481 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:59.120063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5b570-90ea-4118-866c-65287f10e481 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:59.120337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5b570-90ea-4118-866c-65287f10e481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:59.132996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5b570-90ea-4118-866c-65287f10e481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:59.134523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5b570-90ea-4118-866c-65287f10e481 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.148178 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e5a8c-c13f-4ac8-b4f9-0618e173f6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:59.148797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e5a8c-c13f-4ac8-b4f9-0618e173f6be None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:59.149076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e5a8c-c13f-4ac8-b4f9-0618e173f6be None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:59.149350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e5a8c-c13f-4ac8-b4f9-0618e173f6be None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-52e5b570-90ea-4118-866c-65287f10e481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:21:59.149479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:59.150492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:59.151479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:21:59.151479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:21:59.151479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:59.151479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:59.151689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e5a8c-c13f-4ac8-b4f9-0618e173f6be None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:59.151689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e5a8c-c13f-4ac8-b4f9-0618e173f6be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:59.151820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 253/507] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:59.151885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e5a8c-c13f-4ac8-b4f9-0618e173f6be None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.152084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5b570-90ea-4118-866c-65287f10e481 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:59.168810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 255/508] 10.222.0.73 () {56 vars in 1137 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:59.169617 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e5a8c-c13f-4ac8-b4f9-0618e173f6be None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:59.247646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:59.253885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5b570-90ea-4118-866c-65287f10e481 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:59.269963 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e5a8c-c13f-4ac8-b4f9-0618e173f6be None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:59.286977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebf20b-a3fa-405e-870b-d361470be682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:59.287613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebf20b-a3fa-405e-870b-d361470be682 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:59.287940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebf20b-a3fa-405e-870b-d361470be682 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:59.288209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebf20b-a3fa-405e-870b-d361470be682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:59.298721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eebf20b-a3fa-405e-870b-d361470be682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:59.300791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eebf20b-a3fa-405e-870b-d361470be682 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.303520 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0afb08-8611-42df-836b-2a823cee902e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:59.304129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0afb08-8611-42df-836b-2a823cee902e None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:59.304413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0afb08-8611-42df-836b-2a823cee902e None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:59.304688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0afb08-8611-42df-836b-2a823cee902e None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:59.305390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0afb08-8611-42df-836b-2a823cee902e None admin] RBAC: Adding query filter params (name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:59.305695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0afb08-8611-42df-836b-2a823cee902e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:59.307097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0afb08-8611-42df-836b-2a823cee902e None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-3eebf20b-a3fa-405e-870b-d361470be682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:21:59.314494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:59.315393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:59.316841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:21:59.316841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:21:59.316841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:59.316841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:59.317066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 256/509] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:59.317066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0afb08-8611-42df-836b-2a823cee902e None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:59.318280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 254/510] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:59.319300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebf20b-a3fa-405e-870b-d361470be682 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:59.356905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:59.415700 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0afb08-8611-42df-836b-2a823cee902e None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:59.421539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebf20b-a3fa-405e-870b-d361470be682 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:59.449252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0afbf-7f59-41c8-b02f-7d0af85b8bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:59.449861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0afbf-7f59-41c8-b02f-7d0af85b8bf3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:59.450185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0afbf-7f59-41c8-b02f-7d0af85b8bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:59.450462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0afbf-7f59-41c8-b02f-7d0af85b8bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:59.451146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0afbf-7f59-41c8-b02f-7d0af85b8bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:59.451453 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0afbf-7f59-41c8-b02f-7d0af85b8bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:59.452821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0afbf-7f59-41c8-b02f-7d0af85b8bf3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.455076 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3404a-70c0-4bfc-8553-47246fbbd98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:59.455678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3404a-70c0-4bfc-8553-47246fbbd98b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:59.455957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3404a-70c0-4bfc-8553-47246fbbd98b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:59.456271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3404a-70c0-4bfc-8553-47246fbbd98b None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:59.457122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3404a-70c0-4bfc-8553-47246fbbd98b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:59.457423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3404a-70c0-4bfc-8553-47246fbbd98b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:59.458808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3404a-70c0-4bfc-8553-47246fbbd98b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.459886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:59.465055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 257/511] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:59.465944 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0afbf-7f59-41c8-b02f-7d0af85b8bf3 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:59.475621 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 255/512] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:59.475621 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3404a-70c0-4bfc-8553-47246fbbd98b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:59.568166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:59.568447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0afbf-7f59-41c8-b02f-7d0af85b8bf3 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:59.574339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3404a-70c0-4bfc-8553-47246fbbd98b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:59.602652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6e552-9cac-4417-8520-3790d2700f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:59.603262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6e552-9cac-4417-8520-3790d2700f0d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:59.603540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6e552-9cac-4417-8520-3790d2700f0d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:59.603810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6e552-9cac-4417-8520-3790d2700f0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:59.608099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6067581-535e-4dc4-9ff3-67f10ff25ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:59.608723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6067581-535e-4dc4-9ff3-67f10ff25ed0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:59.609002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6067581-535e-4dc4-9ff3-67f10ff25ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:59.609273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6067581-535e-4dc4-9ff3-67f10ff25ed0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:59.620088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6067581-535e-4dc4-9ff3-67f10ff25ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:59.621524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6067581-535e-4dc4-9ff3-67f10ff25ed0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-a6067581-535e-4dc4-9ff3-67f10ff25ed0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:21:59.636843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:59.637777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:59.638658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:21:59.638658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:21:59.638658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:21:59.638658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:59.638658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 256/513] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:59.639324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6067581-535e-4dc4-9ff3-67f10ff25ed0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:59.674247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:59.678057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6e552-9cac-4417-8520-3790d2700f0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:59.679550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6e552-9cac-4417-8520-3790d2700f0d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.741695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6067581-535e-4dc4-9ff3-67f10ff25ed0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:59.773867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24a905-18f6-4a96-8d7a-d15316136f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:59.774504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a905-18f6-4a96-8d7a-d15316136f57 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:59.774789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a905-18f6-4a96-8d7a-d15316136f57 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:59.775111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a905-18f6-4a96-8d7a-d15316136f57 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:59.775745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24a905-18f6-4a96-8d7a-d15316136f57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:21:59.776050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24a905-18f6-4a96-8d7a-d15316136f57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:59.776514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 258/514] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:21:59.777348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6e552-9cac-4417-8520-3790d2700f0d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:59.777440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24a905-18f6-4a96-8d7a-d15316136f57 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.792179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 257/515] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:21:59.792897 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24a905-18f6-4a96-8d7a-d15316136f57 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:59.801501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:59.881578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:21:59.885445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6e552-9cac-4417-8520-3790d2700f0d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:59.893392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24a905-18f6-4a96-8d7a-d15316136f57 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:21:59.919686 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc863f-eb6e-49d8-b2d5-bdf9bcdeeafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:59.920321 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc863f-eb6e-49d8-b2d5-bdf9bcdeeafe None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:59.920600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc863f-eb6e-49d8-b2d5-bdf9bcdeeafe None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:59.920905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc863f-eb6e-49d8-b2d5-bdf9bcdeeafe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:59.927595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f15fc-8b3c-450f-b7f4-207103755f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:21:59.927904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc863f-eb6e-49d8-b2d5-bdf9bcdeeafe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:59.928285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f15fc-8b3c-450f-b7f4-207103755f53 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:21:59.928572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f15fc-8b3c-450f-b7f4-207103755f53 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:21:59.928847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f15fc-8b3c-450f-b7f4-207103755f53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:21:59.929351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc863f-eb6e-49d8-b2d5-bdf9bcdeeafe None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.935767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f15fc-8b3c-450f-b7f4-207103755f53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:21:59.937202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f15fc-8b3c-450f-b7f4-207103755f53 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-04fc863f-eb6e-49d8-b2d5-bdf9bcdeeafe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:21:59.940576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:59.941594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:59.942542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:59.942542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:59.942542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:21:59.942542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:21:59.942542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:21:59.942542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:21:59.942542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:21:59.942542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:21:59.942542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:21:59.942542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:21:59.942542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 259/516] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:59.944195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc863f-eb6e-49d8-b2d5-bdf9bcdeeafe None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-7b8f15fc-8b3c-450f-b7f4-207103755f53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:21:59.950447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:21:59.951337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:21:59.952202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:21:59.952202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:21:59.952202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:21:59.952202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:21:59.952202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:21:59.952202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:21:59.952202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:21:59.952202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:21:59.952202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 258/517] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:21:59.952816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f15fc-8b3c-450f-b7f4-207103755f53 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:00.018157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:00.044701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc863f-eb6e-49d8-b2d5-bdf9bcdeeafe None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:00.054619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f15fc-8b3c-450f-b7f4-207103755f53 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:00.077264 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ce838-b07f-4ab6-adbb-0612da8b9723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:00.077875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ce838-b07f-4ab6-adbb-0612da8b9723 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:00.078191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ce838-b07f-4ab6-adbb-0612da8b9723 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:00.078473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ce838-b07f-4ab6-adbb-0612da8b9723 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:00.079187 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ce838-b07f-4ab6-adbb-0612da8b9723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:00.079491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ce838-b07f-4ab6-adbb-0612da8b9723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:00.080857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ce838-b07f-4ab6-adbb-0612da8b9723 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:00.087867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e5467-7628-411a-ba37-a0522c2fb2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:00.088496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e5467-7628-411a-ba37-a0522c2fb2dc None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:00.088773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e5467-7628-411a-ba37-a0522c2fb2dc None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:00.089044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e5467-7628-411a-ba37-a0522c2fb2dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:00.092912 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 260/518] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:00.093690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ce838-b07f-4ab6-adbb-0612da8b9723 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:00.100482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e5467-7628-411a-ba37-a0522c2fb2dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:00.101959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e5467-7628-411a-ba37-a0522c2fb2dc None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:00.114042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-2b5e5467-7628-411a-ba37-a0522c2fb2dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:00.116374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:00.117199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:00.118088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:00.118088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:00.118088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:00.118088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:00.118088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 259/519] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:00.118349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e5467-7628-411a-ba37-a0522c2fb2dc None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:00.199818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ce838-b07f-4ab6-adbb-0612da8b9723 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:00.224177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e5467-7628-411a-ba37-a0522c2fb2dc None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:00.234110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa935fc-dce0-4db5-8467-4e0f65055ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:00.234729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa935fc-dce0-4db5-8467-4e0f65055ad5 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:00.235010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa935fc-dce0-4db5-8467-4e0f65055ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:00.235287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa935fc-dce0-4db5-8467-4e0f65055ad5 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:00.236224 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa935fc-dce0-4db5-8467-4e0f65055ad5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:00.238000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa935fc-dce0-4db5-8467-4e0f65055ad5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:00.240967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:00.257345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28e2e0-85b1-43d5-99cd-88856b2ec23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:00.257999 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28e2e0-85b1-43d5-99cd-88856b2ec23d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:00.258310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28e2e0-85b1-43d5-99cd-88856b2ec23d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:00.258580 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28e2e0-85b1-43d5-99cd-88856b2ec23d None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:00.259360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28e2e0-85b1-43d5-99cd-88856b2ec23d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:00.259665 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28e2e0-85b1-43d5-99cd-88856b2ec23d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:00.261080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28e2e0-85b1-43d5-99cd-88856b2ec23d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:00.297207 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 260/520] 10.222.0.73 () {56 vars in 1137 bytes} [Thu Aug 4 21:22:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:00.297985 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28e2e0-85b1-43d5-99cd-88856b2ec23d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:00.351440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 261/521] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:22:00 2022] POST /identity/v3/users => generated 247 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:22:00.352354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa935fc-dce0-4db5-8467-4e0f65055ad5 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:00.406691 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28e2e0-85b1-43d5-99cd-88856b2ec23d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:00.446561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b1d7c-19a9-47fc-9402-61060ad2a8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:00.447194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b1d7c-19a9-47fc-9402-61060ad2a8b6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:00.447474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b1d7c-19a9-47fc-9402-61060ad2a8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:00.447747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b1d7c-19a9-47fc-9402-61060ad2a8b6 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:00.448499 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b1d7c-19a9-47fc-9402-61060ad2a8b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:00.448806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b1d7c-19a9-47fc-9402-61060ad2a8b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:00.450322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b1d7c-19a9-47fc-9402-61060ad2a8b6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:00.452638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:00.476604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 261/522] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:00.476604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b1d7c-19a9-47fc-9402-61060ad2a8b6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:00.489094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa935fc-dce0-4db5-8467-4e0f65055ad5 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:00.514602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:00.531129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b3cc4-8462-43f8-8f79-d17541201366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:00.531796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3cc4-8462-43f8-8f79-d17541201366 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:00.532536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3cc4-8462-43f8-8f79-d17541201366 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:00.532536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3cc4-8462-43f8-8f79-d17541201366 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:00.544245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b3cc4-8462-43f8-8f79-d17541201366 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:00.547096 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b3cc4-8462-43f8-8f79-d17541201366 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-b81b3cc4-8462-43f8-8f79-d17541201366 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:00.567557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:00.569296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 262/523] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:00.571081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b3cc4-8462-43f8-8f79-d17541201366 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:00.599010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b1d7c-19a9-47fc-9402-61060ad2a8b6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:00.637699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfac1e-8184-4329-a9ca-a80a87ca1239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:00.638338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfac1e-8184-4329-a9ca-a80a87ca1239 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:00.638616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfac1e-8184-4329-a9ca-a80a87ca1239 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:00.638885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfac1e-8184-4329-a9ca-a80a87ca1239 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:00.639567 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfac1e-8184-4329-a9ca-a80a87ca1239 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:00.639868 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfac1e-8184-4329-a9ca-a80a87ca1239 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:00.641262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfac1e-8184-4329-a9ca-a80a87ca1239 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:00.653495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 262/524] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:00.654251 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfac1e-8184-4329-a9ca-a80a87ca1239 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:00.705247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b3cc4-8462-43f8-8f79-d17541201366 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:00.749847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c13663-9222-4f47-9e2b-2e9c921c7cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:00.750722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c13663-9222-4f47-9e2b-2e9c921c7cd6 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:00.751009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c13663-9222-4f47-9e2b-2e9c921c7cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:00.751282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c13663-9222-4f47-9e2b-2e9c921c7cd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:00.754841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:00.756221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:00.761854 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfac1e-8184-4329-a9ca-a80a87ca1239 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:00.795554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b2587-3a15-47f7-b604-726490e4d6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:00.796168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b2587-3a15-47f7-b604-726490e4d6ca None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:00.796505 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b2587-3a15-47f7-b604-726490e4d6ca None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:00.796874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b2587-3a15-47f7-b604-726490e4d6ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:00.797655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b2587-3a15-47f7-b604-726490e4d6ca None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=e7b8267780364a2181abf5817b54ce5c) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:00.797966 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b2587-3a15-47f7-b604-726490e4d6ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:00.799612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b2587-3a15-47f7-b604-726490e4d6ca None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:00.810202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 263/525] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:00 2022] GET /identity/v3/role_assignments?user.id=e7b8267780364a2181abf5817b54ce5c&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:00.811152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b2587-3a15-47f7-b604-726490e4d6ca None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:00.831831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c13663-9222-4f47-9e2b-2e9c921c7cd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:00.833845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c13663-9222-4f47-9e2b-2e9c921c7cd6 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:00.927663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b2587-3a15-47f7-b604-726490e4d6ca None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:00.955611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 263/526] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:22:00.957620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c13663-9222-4f47-9e2b-2e9c921c7cd6 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:00.967086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652d871-8254-40e5-814b-973b894afcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:00.967952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652d871-8254-40e5-814b-973b894afcbc None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:00.967952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652d871-8254-40e5-814b-973b894afcbc None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:00.968245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652d871-8254-40e5-814b-973b894afcbc None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:00.969038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652d871-8254-40e5-814b-973b894afcbc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:00.969344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652d871-8254-40e5-814b-973b894afcbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:00.970926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652d871-8254-40e5-814b-973b894afcbc None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:00.995397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 264/527] 10.222.0.73 () {56 vars in 1139 bytes} [Thu Aug 4 21:22:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:00.996327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652d871-8254-40e5-814b-973b894afcbc None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:00.999726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:01.017247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:01.087843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c13663-9222-4f47-9e2b-2e9c921c7cd6 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:01.117730 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652d871-8254-40e5-814b-973b894afcbc None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:01.125102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86636230-8710-4a9b-b9cb-b0b30a253bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:01.125894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86636230-8710-4a9b-b9cb-b0b30a253bd2 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:01.126339 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86636230-8710-4a9b-b9cb-b0b30a253bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:01.126710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86636230-8710-4a9b-b9cb-b0b30a253bd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:01.134890 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86636230-8710-4a9b-b9cb-b0b30a253bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:01.136542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86636230-8710-4a9b-b9cb-b0b30a253bd2 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-86636230-8710-4a9b-b9cb-b0b30a253bd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:01.150036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:01.151116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:01.152163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:01.152163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:01.152163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:01.152163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:01.152163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:01.152163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:01.152163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:01.152163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:01.154895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 264/528] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:01.154895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86636230-8710-4a9b-b9cb-b0b30a253bd2 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:01.167844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ebb9a-1a37-4575-9670-6b8c74f3b009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:01.168645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ebb9a-1a37-4575-9670-6b8c74f3b009 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:01.169029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ebb9a-1a37-4575-9670-6b8c74f3b009 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:01.169403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ebb9a-1a37-4575-9670-6b8c74f3b009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:01.180879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ebb9a-1a37-4575-9670-6b8c74f3b009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:01.182478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ebb9a-1a37-4575-9670-6b8c74f3b009 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-5a9ebb9a-1a37-4575-9670-6b8c74f3b009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:01.199502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:01.200595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:01.201645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:01.201645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:01.201645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:01.201645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:01.202927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 265/529] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:01.203917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ebb9a-1a37-4575-9670-6b8c74f3b009 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:01.281616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:01.282970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:01.291992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86636230-8710-4a9b-b9cb-b0b30a253bd2 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:01.317815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ebb9a-1a37-4575-9670-6b8c74f3b009 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:01.343245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad45517-7e37-4fce-8814-081e3f43049a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:01.344381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad45517-7e37-4fce-8814-081e3f43049a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:01.344889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad45517-7e37-4fce-8814-081e3f43049a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:01.345305 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad45517-7e37-4fce-8814-081e3f43049a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:01.359170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27abba9-d4b4-4f78-9b2d-6c1f3b26d74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:01.359956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27abba9-d4b4-4f78-9b2d-6c1f3b26d74d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:01.360336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27abba9-d4b4-4f78-9b2d-6c1f3b26d74d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:01.360961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27abba9-d4b4-4f78-9b2d-6c1f3b26d74d None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:01.361804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27abba9-d4b4-4f78-9b2d-6c1f3b26d74d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:01.362273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27abba9-d4b4-4f78-9b2d-6c1f3b26d74d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:01.364424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27abba9-d4b4-4f78-9b2d-6c1f3b26d74d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:01.382151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 266/530] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:01.382151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27abba9-d4b4-4f78-9b2d-6c1f3b26d74d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:01.446770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad45517-7e37-4fce-8814-081e3f43049a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:01.450110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad45517-7e37-4fce-8814-081e3f43049a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:01.491450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27abba9-d4b4-4f78-9b2d-6c1f3b26d74d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:01.527744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:01.535297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a9607-6cc2-450f-b5f5-2a8ce471998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:01.536043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a9607-6cc2-450f-b5f5-2a8ce471998e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:01.536323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a9607-6cc2-450f-b5f5-2a8ce471998e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:01.536604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a9607-6cc2-450f-b5f5-2a8ce471998e None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:01.537307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a9607-6cc2-450f-b5f5-2a8ce471998e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:01.537684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a9607-6cc2-450f-b5f5-2a8ce471998e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:01.539234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a9607-6cc2-450f-b5f5-2a8ce471998e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:01.553020 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 267/531] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:01.553909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a9607-6cc2-450f-b5f5-2a8ce471998e None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:01.561996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:01.570948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 265/532] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:22:01.572136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad45517-7e37-4fce-8814-081e3f43049a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:01.668339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a9607-6cc2-450f-b5f5-2a8ce471998e None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:01.676961 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad45517-7e37-4fce-8814-081e3f43049a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:01.704506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880d46f-88a5-4aca-8002-374765264d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:01.705176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880d46f-88a5-4aca-8002-374765264d07 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:01.705404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880d46f-88a5-4aca-8002-374765264d07 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:01.705681 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880d46f-88a5-4aca-8002-374765264d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:01.706523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880d46f-88a5-4aca-8002-374765264d07 None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=03a9a6bcfa1b469298774e09a8912b93) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:01.706928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880d46f-88a5-4aca-8002-374765264d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:01.708446 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880d46f-88a5-4aca-8002-374765264d07 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:01.722324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 268/533] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:01 2022] GET /identity/v3/role_assignments?user.id=03a9a6bcfa1b469298774e09a8912b93&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:01.722324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880d46f-88a5-4aca-8002-374765264d07 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:01.722636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accf993-2c9e-499b-bd19-17306c1d8c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:01.722636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accf993-2c9e-499b-bd19-17306c1d8c31 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:01.722636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accf993-2c9e-499b-bd19-17306c1d8c31 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:01.722636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accf993-2c9e-499b-bd19-17306c1d8c31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:01.741677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8accf993-2c9e-499b-bd19-17306c1d8c31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:01.743456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8accf993-2c9e-499b-bd19-17306c1d8c31 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-8accf993-2c9e-499b-bd19-17306c1d8c31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:01.758389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:01.759349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:01.760658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:01.760658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:01.760658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:01.760658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:01.760658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:01.760658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:01.760658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:01.760658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:01.760658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 266/534] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:01.765519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accf993-2c9e-499b-bd19-17306c1d8c31 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:01.834377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:01.895385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:01.908228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accf993-2c9e-499b-bd19-17306c1d8c31 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:01.958037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8531bb-aca5-44bf-b4b2-6c5ff355a5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:01.958037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8531bb-aca5-44bf-b4b2-6c5ff355a5b6 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:01.958037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8531bb-aca5-44bf-b4b2-6c5ff355a5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:01.958037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8531bb-aca5-44bf-b4b2-6c5ff355a5b6 None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:01.958881 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8531bb-aca5-44bf-b4b2-6c5ff355a5b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:01.958881 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8531bb-aca5-44bf-b4b2-6c5ff355a5b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:01.958881 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8531bb-aca5-44bf-b4b2-6c5ff355a5b6 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:01.959641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880d46f-88a5-4aca-8002-374765264d07 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:01.979617 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 267/535] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:01.990036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8531bb-aca5-44bf-b4b2-6c5ff355a5b6 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:02.043695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a2d12-9a94-43b9-9a29-6aa1d5d5ca94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:02.044628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2d12-9a94-43b9-9a29-6aa1d5d5ca94 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:02.045053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2d12-9a94-43b9-9a29-6aa1d5d5ca94 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:02.045473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2d12-9a94-43b9-9a29-6aa1d5d5ca94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:02.078746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a2d12-9a94-43b9-9a29-6aa1d5d5ca94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:02.080250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a2d12-9a94-43b9-9a29-6aa1d5d5ca94 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-5d3a2d12-9a94-43b9-9a29-6aa1d5d5ca94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:02.104399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:02.105387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:02.106847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:02.106847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:02.106847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:02.106847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:02.107071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8531bb-aca5-44bf-b4b2-6c5ff355a5b6 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:02.107142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 269/536] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:02.114858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a2d12-9a94-43b9-9a29-6aa1d5d5ca94 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:02.121424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:02.146469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4c5ae-d799-4d13-9d93-02bbf6bcb227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:02.147238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4c5ae-d799-4d13-9d93-02bbf6bcb227 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:02.147531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4c5ae-d799-4d13-9d93-02bbf6bcb227 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:02.147808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4c5ae-d799-4d13-9d93-02bbf6bcb227 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:02.148593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4c5ae-d799-4d13-9d93-02bbf6bcb227 None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=f3264e17413e4c64b644c1b607cca15b) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:02.148907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4c5ae-d799-4d13-9d93-02bbf6bcb227 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:02.150512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4c5ae-d799-4d13-9d93-02bbf6bcb227 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:02.158842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 268/537] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:01 2022] GET /identity/v3/role_assignments?user.id=f3264e17413e4c64b644c1b607cca15b&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:02.247936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a2d12-9a94-43b9-9a29-6aa1d5d5ca94 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:02.256525 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:02.287566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4780-3802-4228-95dd-1c9823d846c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:02.288200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4780-3802-4228-95dd-1c9823d846c8 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:02.288482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4780-3802-4228-95dd-1c9823d846c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:02.288752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4780-3802-4228-95dd-1c9823d846c8 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:02.289453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4780-3802-4228-95dd-1c9823d846c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:02.289755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4780-3802-4228-95dd-1c9823d846c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:02.291186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4780-3802-4228-95dd-1c9823d846c8 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:02.306381 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 270/538] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:02.311852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4c5ae-d799-4d13-9d93-02bbf6bcb227 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:02.442057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4c5ae-d799-4d13-9d93-02bbf6bcb227 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:02.502901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77065113-c928-4bfe-a573-13b1cfa3496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:02.503543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77065113-c928-4bfe-a573-13b1cfa3496f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:02.503823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77065113-c928-4bfe-a573-13b1cfa3496f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:02.504097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77065113-c928-4bfe-a573-13b1cfa3496f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:02.513583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77065113-c928-4bfe-a573-13b1cfa3496f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:02.515051 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77065113-c928-4bfe-a573-13b1cfa3496f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-77065113-c928-4bfe-a573-13b1cfa3496f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:02.534136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:02.535787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:02.537270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:02.537270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:02.537270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:02.537270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:02.537270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:02.537270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:02.537270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:02.537270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:02.537270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 269/539] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:02.541792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4780-3802-4228-95dd-1c9823d846c8 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:02.572554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54598d-41c5-466a-bce0-4e48ea7fc70b None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:02.572867 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54598d-41c5-466a-bce0-4e48ea7fc70b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:02.573105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54598d-41c5-466a-bce0-4e48ea7fc70b None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:02.574461 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 270/540] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:02.579533 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6a31e-9447-4dca-8c10-4547abdef384 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:02.579813 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6a31e-9447-4dca-8c10-4547abdef384 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:02.580085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6a31e-9447-4dca-8c10-4547abdef384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:02.621615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c6b6a31e-9447-4dca-8c10-4547abdef384 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:02.622904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c6b6a31e-9447-4dca-8c10-4547abdef384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:02.688786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:02.692637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4780-3802-4228-95dd-1c9823d846c8 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:02.739609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f67442-9c54-4546-9462-c63a624cdf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:02.740425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f67442-9c54-4546-9462-c63a624cdf0a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:02.740936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f67442-9c54-4546-9462-c63a624cdf0a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:02.741102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f67442-9c54-4546-9462-c63a624cdf0a None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:02.741723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f67442-9c54-4546-9462-c63a624cdf0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:02.743454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f67442-9c54-4546-9462-c63a624cdf0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:02.744918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f67442-9c54-4546-9462-c63a624cdf0a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:02.761977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 271/541] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:02.768082 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f67442-9c54-4546-9462-c63a624cdf0a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:02.784838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 271/542] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 04 21:22:02.798633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:02.819767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6a31e-9447-4dca-8c10-4547abdef384 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:02.886533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f67442-9c54-4546-9462-c63a624cdf0a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:02.932946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b8d91-8fca-48f0-b208-c36665a04d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:02.935505 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b8d91-8fca-48f0-b208-c36665a04d4a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:02.935997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b8d91-8fca-48f0-b208-c36665a04d4a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:02.936493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b8d91-8fca-48f0-b208-c36665a04d4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:02.937776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b8d91-8fca-48f0-b208-c36665a04d4a None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=c98761f81496456fb3b5a5cffd4ce763) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:02.939556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b8d91-8fca-48f0-b208-c36665a04d4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:02.942170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b8d91-8fca-48f0-b208-c36665a04d4a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:02.954773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 272/543] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:02 2022] GET /identity/v3/role_assignments?user.id=c98761f81496456fb3b5a5cffd4ce763&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:03.008882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6a31e-9447-4dca-8c10-4547abdef384 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:03.054611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258f08d-0766-4380-b7dc-58fc26d9050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:03.055561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258f08d-0766-4380-b7dc-58fc26d9050e None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:03.056025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258f08d-0766-4380-b7dc-58fc26d9050e None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:03.056474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258f08d-0766-4380-b7dc-58fc26d9050e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:03.072008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258f08d-0766-4380-b7dc-58fc26d9050e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-c258f08d-0766-4380-b7dc-58fc26d9050e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:03.077267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:03.078170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:03.078991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:03.086113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 272/544] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:03.087421 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b8d91-8fca-48f0-b208-c36665a04d4a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:03.093543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074bef5-7a5b-4c35-b448-15944995730f None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:03.094075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074bef5-7a5b-4c35-b448-15944995730f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:03.094498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074bef5-7a5b-4c35-b448-15944995730f None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:03.095984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 273/545] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:22:03.106312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b2b30-0d14-4627-9899-898b2abd5e9e None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:03.106816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b2b30-0d14-4627-9899-898b2abd5e9e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:03.107260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b2b30-0d14-4627-9899-898b2abd5e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:03.139065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c20b2b30-0d14-4627-9899-898b2abd5e9e None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:03.140521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c20b2b30-0d14-4627-9899-898b2abd5e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:03.184429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:03.214379 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:03.235390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b8d91-8fca-48f0-b208-c36665a04d4a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:03.273217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 274/546] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:22:03.281373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b5c9b5-2826-4b6b-a932-85642cc50e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:03.282273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5c9b5-2826-4b6b-a932-85642cc50e14 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:03.282783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5c9b5-2826-4b6b-a932-85642cc50e14 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:03.283261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5c9b5-2826-4b6b-a932-85642cc50e14 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:03.284719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5c9b5-2826-4b6b-a932-85642cc50e14 None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:03.285309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5c9b5-2826-4b6b-a932-85642cc50e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:03.286913 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5c9b5-2826-4b6b-a932-85642cc50e14 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:03.296717 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 273/547] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:03.300954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b2b30-0d14-4627-9899-898b2abd5e9e None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:03.305130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b5c9b5-2826-4b6b-a932-85642cc50e14 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:03.468000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b2b30-0d14-4627-9899-898b2abd5e9e None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:03.511752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e211094-534a-42b2-9ade-b9a6d2e3a454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:03.513054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e211094-534a-42b2-9ade-b9a6d2e3a454 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:03.513782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e211094-534a-42b2-9ade-b9a6d2e3a454 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:03.514322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e211094-534a-42b2-9ade-b9a6d2e3a454 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:03.528265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b5c9b5-2826-4b6b-a932-85642cc50e14 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:03.543110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e211094-534a-42b2-9ade-b9a6d2e3a454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-3e211094-534a-42b2-9ade-b9a6d2e3a454 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:03.553534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:03.554810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:03.555656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:03.559256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 275/548] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:03.566497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e211094-534a-42b2-9ade-b9a6d2e3a454 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:03.605593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:03.610036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c1c6d-a843-45fa-a413-b8790aee28d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:03.610036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c1c6d-a843-45fa-a413-b8790aee28d0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:03.610036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c1c6d-a843-45fa-a413-b8790aee28d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:03.610036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c1c6d-a843-45fa-a413-b8790aee28d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:03.639169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c1c6d-a843-45fa-a413-b8790aee28d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:03.640657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c1c6d-a843-45fa-a413-b8790aee28d0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-c86c1c6d-a843-45fa-a413-b8790aee28d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:03.675680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:03.676788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:03.677672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:03.677672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:03.677672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:03.677672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:03.677672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 274/549] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:03.687281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c1c6d-a843-45fa-a413-b8790aee28d0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:03.696864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:03.717060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e211094-534a-42b2-9ade-b9a6d2e3a454 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:03.789083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16bd34-7208-4bab-aace-68d65b45e9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:03.789713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16bd34-7208-4bab-aace-68d65b45e9f6 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:03.789995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16bd34-7208-4bab-aace-68d65b45e9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:03.790291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16bd34-7208-4bab-aace-68d65b45e9f6 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:03.791142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16bd34-7208-4bab-aace-68d65b45e9f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:03.791438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16bd34-7208-4bab-aace-68d65b45e9f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:03.792879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16bd34-7208-4bab-aace-68d65b45e9f6 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:03.802519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 276/550] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:03.810086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16bd34-7208-4bab-aace-68d65b45e9f6 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:03.887163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c1c6d-a843-45fa-a413-b8790aee28d0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:03.906783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:03.919016 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16bd34-7208-4bab-aace-68d65b45e9f6 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:03.955036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f5006-7653-48d7-9fb2-456db77ba108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:03.955681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5006-7653-48d7-9fb2-456db77ba108 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:03.955965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5006-7653-48d7-9fb2-456db77ba108 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:03.956098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ba83c-0f8a-47ed-8c63-8f3638e1aa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:03.956629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5006-7653-48d7-9fb2-456db77ba108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:03.956772 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ba83c-0f8a-47ed-8c63-8f3638e1aa77 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:03.957012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ba83c-0f8a-47ed-8c63-8f3638e1aa77 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:03.957187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ba83c-0f8a-47ed-8c63-8f3638e1aa77 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:03.957883 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ba83c-0f8a-47ed-8c63-8f3638e1aa77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:03.958218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ba83c-0f8a-47ed-8c63-8f3638e1aa77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:03.959603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ba83c-0f8a-47ed-8c63-8f3638e1aa77 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:03.969085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f5006-7653-48d7-9fb2-456db77ba108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:03.970581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f5006-7653-48d7-9fb2-456db77ba108 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:03.985007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 275/551] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-bb3f5006-7653-48d7-9fb2-456db77ba108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:03.988345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:03.989261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:03.991072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:03.991072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:03.991072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:03.991072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:03.991072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 277/552] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:03.991554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ba83c-0f8a-47ed-8c63-8f3638e1aa77 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:03.995073 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f5006-7653-48d7-9fb2-456db77ba108 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:04.083552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:04.105891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f5006-7653-48d7-9fb2-456db77ba108 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:04.139679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:04.146038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2d401-50d8-42d8-8fd3-b4e03d30cc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:04.146989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2d401-50d8-42d8-8fd3-b4e03d30cc78 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.147480 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2d401-50d8-42d8-8fd3-b4e03d30cc78 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.147959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2d401-50d8-42d8-8fd3-b4e03d30cc78 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.148868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2d401-50d8-42d8-8fd3-b4e03d30cc78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:04.149477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2d401-50d8-42d8-8fd3-b4e03d30cc78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:04.151059 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2d401-50d8-42d8-8fd3-b4e03d30cc78 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:04.156862 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ba83c-0f8a-47ed-8c63-8f3638e1aa77 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:04.166078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 278/553] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:04.171645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75db8c-e8a3-4b0a-87d7-e5d7b8582ddc None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.171942 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75db8c-e8a3-4b0a-87d7-e5d7b8582ddc None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.172326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75db8c-e8a3-4b0a-87d7-e5d7b8582ddc None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.174510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 279/554] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:04.175838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75db8c-e8a3-4b0a-87d7-e5d7b8582ddc None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:04.208927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe621c5-71e5-416f-9eca-e49917ae2559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:04.209568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe621c5-71e5-416f-9eca-e49917ae2559 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.209900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe621c5-71e5-416f-9eca-e49917ae2559 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.210224 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe621c5-71e5-416f-9eca-e49917ae2559 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.217887 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe621c5-71e5-416f-9eca-e49917ae2559 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:04.219511 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe621c5-71e5-416f-9eca-e49917ae2559 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-ffe621c5-71e5-416f-9eca-e49917ae2559 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:04.232764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:04.235022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:04.237663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:04.237663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:04.237663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:04.237663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:04.237663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:04.237663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:04.237663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:04.237663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:04.237663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:04.237663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:04.237663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 276/555] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:04.243865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbc8f9-9ef7-41b9-9bac-a4f26536d0bb None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.243865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbc8f9-9ef7-41b9-9bac-a4f26536d0bb None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.243865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbc8f9-9ef7-41b9-9bac-a4f26536d0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.260541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c2cbc8f9-9ef7-41b9-9bac-a4f26536d0bb None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:04.261916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c2cbc8f9-9ef7-41b9-9bac-a4f26536d0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:04.281729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75db8c-e8a3-4b0a-87d7-e5d7b8582ddc None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:04.317224 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab481a7b-b1fd-4dc3-8544-9003e6f61f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:04.317869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab481a7b-b1fd-4dc3-8544-9003e6f61f82 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.318191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab481a7b-b1fd-4dc3-8544-9003e6f61f82 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.318468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab481a7b-b1fd-4dc3-8544-9003e6f61f82 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.326111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab481a7b-b1fd-4dc3-8544-9003e6f61f82 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:04.327578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab481a7b-b1fd-4dc3-8544-9003e6f61f82 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:04.334712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 277/556] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:04.335834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbc8f9-9ef7-41b9-9bac-a4f26536d0bb None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-ab481a7b-b1fd-4dc3-8544-9003e6f61f82 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:04.339699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:04.340681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:04.342567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:04.342567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:04.342567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:04.342567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:04.342567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:04.342567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:04.342567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:04.342567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:04.342567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:04.342567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:04.342567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 280/557] 10.222.0.73 () {56 vars in 1093 bytes} [Thu Aug 4 21:22:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:04.346602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab481a7b-b1fd-4dc3-8544-9003e6f61f82 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:04.400785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:04.402659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:04.459720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbc8f9-9ef7-41b9-9bac-a4f26536d0bb None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:04.463923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab481a7b-b1fd-4dc3-8544-9003e6f61f82 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:04.501745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b6c31-40e7-4355-a1c0-fd7b2f8f6537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:04.502399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b6c31-40e7-4355-a1c0-fd7b2f8f6537 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.502681 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b6c31-40e7-4355-a1c0-fd7b2f8f6537 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.503008 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b6c31-40e7-4355-a1c0-fd7b2f8f6537 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.503875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b6c31-40e7-4355-a1c0-fd7b2f8f6537 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:04.504189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b6c31-40e7-4355-a1c0-fd7b2f8f6537 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:04.504775 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f640761-f494-4478-b879-5d6ec4e95b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:04.505392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f640761-f494-4478-b879-5d6ec4e95b70 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.505680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f640761-f494-4478-b879-5d6ec4e95b70 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.505784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b6c31-40e7-4355-a1c0-fd7b2f8f6537 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:04.505954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f640761-f494-4478-b879-5d6ec4e95b70 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.506780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f640761-f494-4478-b879-5d6ec4e95b70 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:04.507151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f640761-f494-4478-b879-5d6ec4e95b70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:04.509623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f640761-f494-4478-b879-5d6ec4e95b70 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:04.531083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 281/558] 10.222.0.73 () {56 vars in 1133 bytes} [Thu Aug 4 21:22:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:04.532008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f640761-f494-4478-b879-5d6ec4e95b70 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:04.539202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 278/559] 10.222.0.73 () {56 vars in 1135 bytes} [Thu Aug 4 21:22:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:04.540121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b6c31-40e7-4355-a1c0-fd7b2f8f6537 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:04.645098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f640761-f494-4478-b879-5d6ec4e95b70 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:04.654491 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b6c31-40e7-4355-a1c0-fd7b2f8f6537 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:04.656499 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:04.668203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:04.682637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994bb18-de9f-4e44-a3b3-cf5cbaf326e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:04.683288 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994bb18-de9f-4e44-a3b3-cf5cbaf326e2 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.683575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994bb18-de9f-4e44-a3b3-cf5cbaf326e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.683857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994bb18-de9f-4e44-a3b3-cf5cbaf326e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.691042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5f65d-f81d-4dae-b0d5-e776c05de47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:04.692072 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5f65d-f81d-4dae-b0d5-e776c05de47a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.692571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5f65d-f81d-4dae-b0d5-e776c05de47a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.693027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5f65d-f81d-4dae-b0d5-e776c05de47a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.694959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994bb18-de9f-4e44-a3b3-cf5cbaf326e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-9994bb18-de9f-4e44-a3b3-cf5cbaf326e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:04.699885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:04.701015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:04.702072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:04.710120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 282/560] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:04.710120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994bb18-de9f-4e44-a3b3-cf5cbaf326e2 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:04.710836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5f65d-f81d-4dae-b0d5-e776c05de47a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:04.712284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5f65d-f81d-4dae-b0d5-e776c05de47a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-73a5f65d-f81d-4dae-b0d5-e776c05de47a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:04.728121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:04.729322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:04.730502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:04.730502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:04.730502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:04.730502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:04.733477 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 279/561] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:04.734703 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dbefb-d778-44e4-99c9-765e9e2f32ac None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.734989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dbefb-d778-44e4-99c9-765e9e2f32ac None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.735267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dbefb-d778-44e4-99c9-765e9e2f32ac None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.736891 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 280/562] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:04.737837 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dbefb-d778-44e4-99c9-765e9e2f32ac None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:04.832448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994bb18-de9f-4e44-a3b3-cf5cbaf326e2 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:04.852540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dbefb-d778-44e4-99c9-765e9e2f32ac None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:04.867839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cf18a-501f-4479-8b1f-97eda3d1b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:04.868452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cf18a-501f-4479-8b1f-97eda3d1b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.868740 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cf18a-501f-4479-8b1f-97eda3d1b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.869015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cf18a-501f-4479-8b1f-97eda3d1b6f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.893220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5dda5-1d13-4846-9d18-0358376ca7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:04.894496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5dda5-1d13-4846-9d18-0358376ca7c7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:04.894946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5dda5-1d13-4846-9d18-0358376ca7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:04.895292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5dda5-1d13-4846-9d18-0358376ca7c7 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:04.895960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5dda5-1d13-4846-9d18-0358376ca7c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:04.896268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5dda5-1d13-4846-9d18-0358376ca7c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:04.897633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5dda5-1d13-4846-9d18-0358376ca7c7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:04.908323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 281/563] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:04.909764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5dda5-1d13-4846-9d18-0358376ca7c7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:04.920853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:04.945811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:04.949854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cf18a-501f-4479-8b1f-97eda3d1b6f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:04.951411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cf18a-501f-4479-8b1f-97eda3d1b6f1 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:05.021341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5dda5-1d13-4846-9d18-0358376ca7c7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:05.056492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763052f-a990-4854-aac2-1e627b396729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:05.061821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 283/564] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:22:05.062052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763052f-a990-4854-aac2-1e627b396729 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.062052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763052f-a990-4854-aac2-1e627b396729 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.062052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763052f-a990-4854-aac2-1e627b396729 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.062313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc5c10-d9b3-473d-93d0-bd38cc94f669 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.062398 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763052f-a990-4854-aac2-1e627b396729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:05.062479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc5c10-d9b3-473d-93d0-bd38cc94f669 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.062587 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763052f-a990-4854-aac2-1e627b396729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:05.062670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc5c10-d9b3-473d-93d0-bd38cc94f669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.062748 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763052f-a990-4854-aac2-1e627b396729 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:05.074595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 282/565] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:05.077200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763052f-a990-4854-aac2-1e627b396729 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:05.084363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a5cc5c10-d9b3-473d-93d0-bd38cc94f669 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:05.085571 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a5cc5c10-d9b3-473d-93d0-bd38cc94f669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:05.152121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:05.156904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 284/566] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:05.157710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc5c10-d9b3-473d-93d0-bd38cc94f669 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:05.180293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:05.195845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763052f-a990-4854-aac2-1e627b396729 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:05.231281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc76dd-3844-495d-9f88-bbb689803131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:05.232040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc76dd-3844-495d-9f88-bbb689803131 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.232393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc76dd-3844-495d-9f88-bbb689803131 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.232733 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc76dd-3844-495d-9f88-bbb689803131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.246834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc76dd-3844-495d-9f88-bbb689803131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:05.248390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc76dd-3844-495d-9f88-bbb689803131 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-34cc76dd-3844-495d-9f88-bbb689803131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:05.264480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:05.265577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:05.266704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:05.266704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:05.266704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:05.266704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:05.267402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 283/567] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:05.267501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc5c10-d9b3-473d-93d0-bd38cc94f669 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:05.268004 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc76dd-3844-495d-9f88-bbb689803131 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:05.307956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d849e-4438-453f-9cd9-83101dae1cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:05.308690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d849e-4438-453f-9cd9-83101dae1cb8 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.309056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d849e-4438-453f-9cd9-83101dae1cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.309393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d849e-4438-453f-9cd9-83101dae1cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.320260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d849e-4438-453f-9cd9-83101dae1cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:05.321737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d849e-4438-453f-9cd9-83101dae1cb8 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-709d849e-4438-453f-9cd9-83101dae1cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:05.336950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:05.338065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:05.339161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:05.339161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:05.339161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:05.339161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:05.342477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 285/568] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:05.343288 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d849e-4438-453f-9cd9-83101dae1cb8 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:05.383002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc76dd-3844-495d-9f88-bbb689803131 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:05.398401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:05.422884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343796e-94b1-447d-bf51-4558580c13f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:05.423454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343796e-94b1-447d-bf51-4558580c13f0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.423739 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343796e-94b1-447d-bf51-4558580c13f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.424015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343796e-94b1-447d-bf51-4558580c13f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.434379 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343796e-94b1-447d-bf51-4558580c13f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:05.435885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343796e-94b1-447d-bf51-4558580c13f0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:05.438264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-2343796e-94b1-447d-bf51-4558580c13f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:05.450027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:05.456661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:05.457919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:05.457919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:05.457919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:05.457919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:05.457919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:05.457919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:05.457919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:05.457919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:05.458486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d849e-4438-453f-9cd9-83101dae1cb8 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:05.458586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 284/569] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:05.458586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343796e-94b1-447d-bf51-4558580c13f0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:05.493022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a24fdd-f5d9-4a01-90d6-92a35b333139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:05.493677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24fdd-f5d9-4a01-90d6-92a35b333139 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.493962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24fdd-f5d9-4a01-90d6-92a35b333139 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.494319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24fdd-f5d9-4a01-90d6-92a35b333139 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.505390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a24fdd-f5d9-4a01-90d6-92a35b333139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-62a24fdd-f5d9-4a01-90d6-92a35b333139 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:05.510884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:05.511801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:05.512746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 286/570] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:05.513458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a24fdd-f5d9-4a01-90d6-92a35b333139 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:05.569041 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343796e-94b1-447d-bf51-4558580c13f0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:05.602578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c4eb6-d0b7-4f4e-b7fc-a874ad8b64cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:05.603185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c4eb6-d0b7-4f4e-b7fc-a874ad8b64cd None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.603467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c4eb6-d0b7-4f4e-b7fc-a874ad8b64cd None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.603747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c4eb6-d0b7-4f4e-b7fc-a874ad8b64cd None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.604431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c4eb6-d0b7-4f4e-b7fc-a874ad8b64cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:05.604963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c4eb6-d0b7-4f4e-b7fc-a874ad8b64cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:05.606117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c4eb6-d0b7-4f4e-b7fc-a874ad8b64cd None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:05.617860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 285/571] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:05.618683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c4eb6-d0b7-4f4e-b7fc-a874ad8b64cd None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:05.621801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a24fdd-f5d9-4a01-90d6-92a35b333139 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:05.635056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:05.656274 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f350993-675a-49b6-8ca0-f3b45797d4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:05.656883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f350993-675a-49b6-8ca0-f3b45797d4a2 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.657163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f350993-675a-49b6-8ca0-f3b45797d4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.657435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f350993-675a-49b6-8ca0-f3b45797d4a2 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.658260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f350993-675a-49b6-8ca0-f3b45797d4a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:05.658566 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f350993-675a-49b6-8ca0-f3b45797d4a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:05.660158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f350993-675a-49b6-8ca0-f3b45797d4a2 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:05.672283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 287/572] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:05.673122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f350993-675a-49b6-8ca0-f3b45797d4a2 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:05.682471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:05.723678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c4eb6-d0b7-4f4e-b7fc-a874ad8b64cd None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:05.757442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5866c-8acb-45ac-b6fd-64761040da67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:05.758074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5866c-8acb-45ac-b6fd-64761040da67 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.758357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5866c-8acb-45ac-b6fd-64761040da67 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.758793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5866c-8acb-45ac-b6fd-64761040da67 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.759373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5866c-8acb-45ac-b6fd-64761040da67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:05.759672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5866c-8acb-45ac-b6fd-64761040da67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:05.761071 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5866c-8acb-45ac-b6fd-64761040da67 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:05.772521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f350993-675a-49b6-8ca0-f3b45797d4a2 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:05.773120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 286/573] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:05.774389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952eb0b-4f3a-484c-931e-c46c4b63f5ff None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.774675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952eb0b-4f3a-484c-931e-c46c4b63f5ff None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.774949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952eb0b-4f3a-484c-931e-c46c4b63f5ff None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.776424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 287/574] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:05.777122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952eb0b-4f3a-484c-931e-c46c4b63f5ff None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:05.807406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee823431-5c68-4a35-a174-747a2028a6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:05.808018 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee823431-5c68-4a35-a174-747a2028a6fe None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.808299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee823431-5c68-4a35-a174-747a2028a6fe None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.808574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee823431-5c68-4a35-a174-747a2028a6fe None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.809283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee823431-5c68-4a35-a174-747a2028a6fe None admin] RBAC: Adding query filter params (name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:05.809632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee823431-5c68-4a35-a174-747a2028a6fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:05.811165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee823431-5c68-4a35-a174-747a2028a6fe None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:05.820592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 288/575] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:05.821385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee823431-5c68-4a35-a174-747a2028a6fe None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:05.854546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:05.876259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952eb0b-4f3a-484c-931e-c46c4b63f5ff None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:05.894113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:05.910715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23143ac2-8b6b-4e39-a8fb-4489d7ce7ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:05.911348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23143ac2-8b6b-4e39-a8fb-4489d7ce7ace None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.911634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23143ac2-8b6b-4e39-a8fb-4489d7ce7ace None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.911907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23143ac2-8b6b-4e39-a8fb-4489d7ce7ace None admin] PATH_INFO: `/v3/users/glance` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.920764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee823431-5c68-4a35-a174-747a2028a6fe None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:05.927460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23143ac2-8b6b-4e39-a8fb-4489d7ce7ace None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:05.929065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23143ac2-8b6b-4e39-a8fb-4489d7ce7ace None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-23143ac2-8b6b-4e39-a8fb-4489d7ce7ace None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:05.942147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:05.943300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 288/576] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:22:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:05.944255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23143ac2-8b6b-4e39-a8fb-4489d7ce7ace None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:05.959896 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048db91-a4cd-4d53-9079-64f1b6a677b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:05.960520 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048db91-a4cd-4d53-9079-64f1b6a677b4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.960799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048db91-a4cd-4d53-9079-64f1b6a677b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.961070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048db91-a4cd-4d53-9079-64f1b6a677b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:05.967780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048db91-a4cd-4d53-9079-64f1b6a677b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:05.969273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048db91-a4cd-4d53-9079-64f1b6a677b4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-a048db91-a4cd-4d53-9079-64f1b6a677b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:05.980506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:05.981568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:05.982583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:05.982583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:05.982583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:05.982583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:05.982583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:05.982583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:05.982583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:05.982583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:05.982583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 289/577] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:05.983069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c4ad8-1867-4805-afde-7f3a08e8f0b8 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:05.983395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c4ad8-1867-4805-afde-7f3a08e8f0b8 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:05.983616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c4ad8-1867-4805-afde-7f3a08e8f0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:06.001330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1f6c4ad8-1867-4805-afde-7f3a08e8f0b8 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:06.002548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1f6c4ad8-1867-4805-afde-7f3a08e8f0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:06.054336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23143ac2-8b6b-4e39-a8fb-4489d7ce7ace None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:06.068197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 290/578] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:06.069017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c4ad8-1867-4805-afde-7f3a08e8f0b8 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:06.087862 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935eff7-91f9-478f-a037-3654f77331c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:06.088542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935eff7-91f9-478f-a037-3654f77331c1 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:06.088828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935eff7-91f9-478f-a037-3654f77331c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:06.089107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935eff7-91f9-478f-a037-3654f77331c1 None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/e7b8267780364a2181abf5817b54ce5c/roles/280f24425d0a4b16951d30c20d7290f9` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:06.093922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:06.104818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935eff7-91f9-478f-a037-3654f77331c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=e7b8267780364a2181abf5817b54ce5c, role_id=280f24425d0a4b16951d30c20d7290f9)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:06.106292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935eff7-91f9-478f-a037-3654f77331c1 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:06.163910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 289/579] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:22:05 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/e7b8267780364a2181abf5817b54ce5c/roles/280f24425d0a4b16951d30c20d7290f9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:22:06.164729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935eff7-91f9-478f-a037-3654f77331c1 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:06.173106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c4ad8-1867-4805-afde-7f3a08e8f0b8 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:06.198591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:06.214403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19212b-680b-4f44-a591-f3d72f1f947f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:06.215236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19212b-680b-4f44-a591-f3d72f1f947f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:06.215589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19212b-680b-4f44-a591-f3d72f1f947f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:06.215943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19212b-680b-4f44-a591-f3d72f1f947f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:06.228258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19212b-680b-4f44-a591-f3d72f1f947f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:06.229826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19212b-680b-4f44-a591-f3d72f1f947f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-3a19212b-680b-4f44-a591-f3d72f1f947f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:06.247038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:06.248231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:06.249335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:06.249335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:06.249335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:06.249335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:06.254313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 291/580] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:06.255225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19212b-680b-4f44-a591-f3d72f1f947f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:06.289536 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935eff7-91f9-478f-a037-3654f77331c1 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:06.330926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf912a-4dbd-4fc8-a242-86f9cfd32e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:06.331748 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf912a-4dbd-4fc8-a242-86f9cfd32e3d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:06.332143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf912a-4dbd-4fc8-a242-86f9cfd32e3d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:06.332544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf912a-4dbd-4fc8-a242-86f9cfd32e3d None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:06.333441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf912a-4dbd-4fc8-a242-86f9cfd32e3d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:06.333764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf912a-4dbd-4fc8-a242-86f9cfd32e3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:06.335295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf912a-4dbd-4fc8-a242-86f9cfd32e3d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:06.350764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:06.353686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 290/581] 10.222.0.73 () {56 vars in 1137 bytes} [Thu Aug 4 21:22:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:06.354664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf912a-4dbd-4fc8-a242-86f9cfd32e3d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:06.378881 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19212b-680b-4f44-a591-f3d72f1f947f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:06.416396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd27c5-109b-42ad-b57a-2e90e2dd751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:06.417070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd27c5-109b-42ad-b57a-2e90e2dd751e None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:06.417366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd27c5-109b-42ad-b57a-2e90e2dd751e None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:06.417637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd27c5-109b-42ad-b57a-2e90e2dd751e None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:06.418417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd27c5-109b-42ad-b57a-2e90e2dd751e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:06.418725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd27c5-109b-42ad-b57a-2e90e2dd751e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:06.420161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd27c5-109b-42ad-b57a-2e90e2dd751e None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:06.432421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 292/582] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:06.433237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd27c5-109b-42ad-b57a-2e90e2dd751e None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:06.456714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:06.471228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf912a-4dbd-4fc8-a242-86f9cfd32e3d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:06.508467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345cdf0-068e-4756-acb6-1da3b2955b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:06.509145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345cdf0-068e-4756-acb6-1da3b2955b99 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:06.509429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345cdf0-068e-4756-acb6-1da3b2955b99 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:06.509728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345cdf0-068e-4756-acb6-1da3b2955b99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:06.521706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345cdf0-068e-4756-acb6-1da3b2955b99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-6345cdf0-068e-4756-acb6-1da3b2955b99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:06.526371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:06.527488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 291/583] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:06.533105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345cdf0-068e-4756-acb6-1da3b2955b99 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:06.554744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd27c5-109b-42ad-b57a-2e90e2dd751e None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:06.591597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff1a37-d9d2-447f-bd30-d20a9ab1ddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:06.592354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff1a37-d9d2-447f-bd30-d20a9ab1ddb0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:06.592731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff1a37-d9d2-447f-bd30-d20a9ab1ddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:06.593109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff1a37-d9d2-447f-bd30-d20a9ab1ddb0 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:06.593905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff1a37-d9d2-447f-bd30-d20a9ab1ddb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:06.594335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff1a37-d9d2-447f-bd30-d20a9ab1ddb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:06.595798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff1a37-d9d2-447f-bd30-d20a9ab1ddb0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:06.598504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:06.612818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 293/584] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:06.612818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff1a37-d9d2-447f-bd30-d20a9ab1ddb0 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:06.640873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345cdf0-068e-4756-acb6-1da3b2955b99 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:06.678890 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5d26e-0709-47c7-bb14-e56ec78f78cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:06.679528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5d26e-0709-47c7-bb14-e56ec78f78cb None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:06.679806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5d26e-0709-47c7-bb14-e56ec78f78cb None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:06.680078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5d26e-0709-47c7-bb14-e56ec78f78cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:06.690750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5d26e-0709-47c7-bb14-e56ec78f78cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:06.692192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5d26e-0709-47c7-bb14-e56ec78f78cb None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:06.699875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-7cf5d26e-0709-47c7-bb14-e56ec78f78cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:06.710400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:06.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:06.712698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:06.712698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:06.712698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:06.712698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:06.712698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 292/585] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:06.715138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5d26e-0709-47c7-bb14-e56ec78f78cb None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:06.719328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff1a37-d9d2-447f-bd30-d20a9ab1ddb0 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:06.756481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122dfd6-3d32-4971-be5b-427dbdf54f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:06.757179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122dfd6-3d32-4971-be5b-427dbdf54f15 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:06.757531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122dfd6-3d32-4971-be5b-427dbdf54f15 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:06.757876 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122dfd6-3d32-4971-be5b-427dbdf54f15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:06.758764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122dfd6-3d32-4971-be5b-427dbdf54f15 None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=3342e1039131496ba121b04685ec7b3d) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:06.759202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122dfd6-3d32-4971-be5b-427dbdf54f15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:06.760619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122dfd6-3d32-4971-be5b-427dbdf54f15 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:06.766297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 294/586] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:06 2022] GET /identity/v3/role_assignments?user.id=3342e1039131496ba121b04685ec7b3d&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:06.766928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122dfd6-3d32-4971-be5b-427dbdf54f15 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:06.846818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5d26e-0709-47c7-bb14-e56ec78f78cb None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:06.852691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:06.890459 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0883c1-68f1-4569-9169-833c746cd68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:06.891588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0883c1-68f1-4569-9169-833c746cd68f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:06.892511 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0883c1-68f1-4569-9169-833c746cd68f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:06.892943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0883c1-68f1-4569-9169-833c746cd68f None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:06.894091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0883c1-68f1-4569-9169-833c746cd68f None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:06.894672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0883c1-68f1-4569-9169-833c746cd68f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:06.896977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0883c1-68f1-4569-9169-833c746cd68f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:06.914047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122dfd6-3d32-4971-be5b-427dbdf54f15 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:06.914557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 293/587] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:06.916568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0883c1-68f1-4569-9169-833c746cd68f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:06.954387 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065a8fa-9541-4907-ba4d-776cda40af1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:06.955453 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065a8fa-9541-4907-ba4d-776cda40af1d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:06.955905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065a8fa-9541-4907-ba4d-776cda40af1d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:06.956511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065a8fa-9541-4907-ba4d-776cda40af1d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:06.965045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065a8fa-9541-4907-ba4d-776cda40af1d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:06.966456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065a8fa-9541-4907-ba4d-776cda40af1d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-3065a8fa-9541-4907-ba4d-776cda40af1d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:06.978642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:06.979831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:06.980862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:06.980862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:06.980862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:06.980862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:06.980862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:06.980862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:06.980862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:06.980862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:06.980862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:22:06.980862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:06.984632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 295/588] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:22:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:06.985490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065a8fa-9541-4907-ba4d-776cda40af1d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:07.014689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:07.062554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0883c1-68f1-4569-9169-833c746cd68f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:07.101875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065a8fa-9541-4907-ba4d-776cda40af1d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:07.106972 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1b847-3309-470c-8fa7-7f4a08ddf567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:07.107785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1b847-3309-470c-8fa7-7f4a08ddf567 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:07.108218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1b847-3309-470c-8fa7-7f4a08ddf567 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:07.108640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1b847-3309-470c-8fa7-7f4a08ddf567 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:07.110110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1b847-3309-470c-8fa7-7f4a08ddf567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:07.110577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1b847-3309-470c-8fa7-7f4a08ddf567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:07.113758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1b847-3309-470c-8fa7-7f4a08ddf567 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:07.123808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:07.136005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 294/589] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:07.136005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1b847-3309-470c-8fa7-7f4a08ddf567 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:07.148948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67057cf-9afb-4208-8891-0a2f3c42968c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:07.149824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67057cf-9afb-4208-8891-0a2f3c42968c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:07.150751 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67057cf-9afb-4208-8891-0a2f3c42968c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:07.151177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67057cf-9afb-4208-8891-0a2f3c42968c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:07.166825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67057cf-9afb-4208-8891-0a2f3c42968c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:07.169376 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67057cf-9afb-4208-8891-0a2f3c42968c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-d67057cf-9afb-4208-8891-0a2f3c42968c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:07.192010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:07.193545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:07.194705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:07.194705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:07.194705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:07.194705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:07.201340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 296/590] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:07.201340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67057cf-9afb-4208-8891-0a2f3c42968c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:07.254145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1b847-3309-470c-8fa7-7f4a08ddf567 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:07.277865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:07.294493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf6b3d-712e-4d9a-bf01-6a6adfe58015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:07.295242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf6b3d-712e-4d9a-bf01-6a6adfe58015 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:07.295597 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf6b3d-712e-4d9a-bf01-6a6adfe58015 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:07.295951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf6b3d-712e-4d9a-bf01-6a6adfe58015 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:07.296912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf6b3d-712e-4d9a-bf01-6a6adfe58015 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:07.297292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf6b3d-712e-4d9a-bf01-6a6adfe58015 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:07.298923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf6b3d-712e-4d9a-bf01-6a6adfe58015 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:07.322473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 295/591] 10.222.0.73 () {56 vars in 1137 bytes} [Thu Aug 4 21:22:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:07.322473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf6b3d-712e-4d9a-bf01-6a6adfe58015 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:07.328221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67057cf-9afb-4208-8891-0a2f3c42968c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:07.366719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6755095-2831-4ead-ac38-303f4b7a93ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:07.367666 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755095-2831-4ead-ac38-303f4b7a93ff None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:07.368163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755095-2831-4ead-ac38-303f4b7a93ff None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:07.368629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755095-2831-4ead-ac38-303f4b7a93ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:07.377003 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6755095-2831-4ead-ac38-303f4b7a93ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:07.378858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6755095-2831-4ead-ac38-303f4b7a93ff None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-f6755095-2831-4ead-ac38-303f4b7a93ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:07.392367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:07.393297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:07.394834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:07.394834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:07.394834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:07.394834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:07.394834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:07.394834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:07.394834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:07.394834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:07.401672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 297/592] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:07.402845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6755095-2831-4ead-ac38-303f4b7a93ff None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:07.429138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf6b3d-712e-4d9a-bf01-6a6adfe58015 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:07.438621 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:07.467498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedcbb3-658f-454a-8cd3-2b3a8487a2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:07.468483 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedcbb3-658f-454a-8cd3-2b3a8487a2df None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:07.469000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedcbb3-658f-454a-8cd3-2b3a8487a2df None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:07.469480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedcbb3-658f-454a-8cd3-2b3a8487a2df None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:07.470423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedcbb3-658f-454a-8cd3-2b3a8487a2df None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:07.470937 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedcbb3-658f-454a-8cd3-2b3a8487a2df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:07.473175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedcbb3-658f-454a-8cd3-2b3a8487a2df None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:07.490389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 296/593] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:07.491643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedcbb3-658f-454a-8cd3-2b3a8487a2df None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:07.520846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6755095-2831-4ead-ac38-303f4b7a93ff None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:07.547323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:07.570881 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dda4c-ebc0-4177-b1dd-a2dadc627f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:07.571899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dda4c-ebc0-4177-b1dd-a2dadc627f44 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:07.572250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dda4c-ebc0-4177-b1dd-a2dadc627f44 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:07.572590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dda4c-ebc0-4177-b1dd-a2dadc627f44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:07.587271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5dda4c-ebc0-4177-b1dd-a2dadc627f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:07.588950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5dda4c-ebc0-4177-b1dd-a2dadc627f44 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-af5dda4c-ebc0-4177-b1dd-a2dadc627f44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:07.609334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:07.610522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:07.611780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:07.611780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:07.611780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:07.611780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:07.612910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 298/594] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:07.614206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dda4c-ebc0-4177-b1dd-a2dadc627f44 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:07.620266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedcbb3-658f-454a-8cd3-2b3a8487a2df None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:07.663941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb617958-3bcd-4647-922c-840a6eadf8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:07.664594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb617958-3bcd-4647-922c-840a6eadf8ed None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:07.665592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb617958-3bcd-4647-922c-840a6eadf8ed None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:07.665592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb617958-3bcd-4647-922c-840a6eadf8ed None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:07.666119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb617958-3bcd-4647-922c-840a6eadf8ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:07.666432 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb617958-3bcd-4647-922c-840a6eadf8ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:07.667872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb617958-3bcd-4647-922c-840a6eadf8ed None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:07.685633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 297/595] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:07.686652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb617958-3bcd-4647-922c-840a6eadf8ed None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:07.719030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:07.736219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dda4c-ebc0-4177-b1dd-a2dadc627f44 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:07.773974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65f4dc-b02a-4150-8fac-9408736e4082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:07.774972 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65f4dc-b02a-4150-8fac-9408736e4082 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:07.775468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65f4dc-b02a-4150-8fac-9408736e4082 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:07.775965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65f4dc-b02a-4150-8fac-9408736e4082 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:07.783904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65f4dc-b02a-4150-8fac-9408736e4082 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:07.785671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65f4dc-b02a-4150-8fac-9408736e4082 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:07.795839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-ce65f4dc-b02a-4150-8fac-9408736e4082 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:07.798111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:07.799035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:07.800089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:07.800089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:07.800089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:07.800089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:07.800089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:07.800089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:07.800089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:07.800089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:07.800089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:22:07.800089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:07.800649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb617958-3bcd-4647-922c-840a6eadf8ed None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:07.806272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 299/596] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:07.807368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65f4dc-b02a-4150-8fac-9408736e4082 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:07.838504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18877702-e401-4f60-b9b4-1dc73b421aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:07.839120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18877702-e401-4f60-b9b4-1dc73b421aed None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:07.839404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18877702-e401-4f60-b9b4-1dc73b421aed None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:07.839680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18877702-e401-4f60-b9b4-1dc73b421aed None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:07.840373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18877702-e401-4f60-b9b4-1dc73b421aed None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:07.840679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18877702-e401-4f60-b9b4-1dc73b421aed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:07.842321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18877702-e401-4f60-b9b4-1dc73b421aed None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:07.855171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 298/597] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:07.856019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18877702-e401-4f60-b9b4-1dc73b421aed None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:07.921575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65f4dc-b02a-4150-8fac-9408736e4082 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:07.958468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bab045-bf0d-4c81-bdb7-78431c6c2983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:07.959495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bab045-bf0d-4c81-bdb7-78431c6c2983 None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:07.959978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bab045-bf0d-4c81-bdb7-78431c6c2983 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:07.960430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bab045-bf0d-4c81-bdb7-78431c6c2983 None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/03a9a6bcfa1b469298774e09a8912b93/roles/280f24425d0a4b16951d30c20d7290f9` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:07.975870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18877702-e401-4f60-b9b4-1dc73b421aed None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:07.977363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bab045-bf0d-4c81-bdb7-78431c6c2983 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=03a9a6bcfa1b469298774e09a8912b93, role_id=280f24425d0a4b16951d30c20d7290f9)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:07.979365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bab045-bf0d-4c81-bdb7-78431c6c2983 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:07.981844 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:08.020392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a9387-b2f9-48af-be81-f6f4f007db07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:08.021015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a9387-b2f9-48af-be81-f6f4f007db07 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:08.021299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a9387-b2f9-48af-be81-f6f4f007db07 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:08.021628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a9387-b2f9-48af-be81-f6f4f007db07 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:08.022475 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a9387-b2f9-48af-be81-f6f4f007db07 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:08.022787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a9387-b2f9-48af-be81-f6f4f007db07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:08.024392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a9387-b2f9-48af-be81-f6f4f007db07 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:08.051613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 299/598] 10.222.0.73 () {56 vars in 1139 bytes} [Thu Aug 4 21:22:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:08.053245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a9387-b2f9-48af-be81-f6f4f007db07 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:08.058316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 300/599] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:22:07 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/03a9a6bcfa1b469298774e09a8912b93/roles/280f24425d0a4b16951d30c20d7290f9 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:22:08.067283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bab045-bf0d-4c81-bdb7-78431c6c2983 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:08.080467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:08.185160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a9387-b2f9-48af-be81-f6f4f007db07 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:08.225850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265544d8-c1bc-4f32-99e3-5466522b908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:08.226539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265544d8-c1bc-4f32-99e3-5466522b908e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:08.226823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265544d8-c1bc-4f32-99e3-5466522b908e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:08.227100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265544d8-c1bc-4f32-99e3-5466522b908e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:08.234734 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265544d8-c1bc-4f32-99e3-5466522b908e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:08.236471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265544d8-c1bc-4f32-99e3-5466522b908e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-265544d8-c1bc-4f32-99e3-5466522b908e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:08.250034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:08.250990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:08.251893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:08.251893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:08.251893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:08.251893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:08.251893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:08.251893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:08.251893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:08.251893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:08.251893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 300/600] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:08.258032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b63d-d04b-49c7-b1cb-13be06cb6688 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:08.258032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b63d-d04b-49c7-b1cb-13be06cb6688 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:08.258032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b63d-d04b-49c7-b1cb-13be06cb6688 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:08.258032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 301/601] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:08.259158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704b63d-d04b-49c7-b1cb-13be06cb6688 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:08.266113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bab045-bf0d-4c81-bdb7-78431c6c2983 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:08.313199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200eef30-76b8-42ae-af2f-fd7fcba092fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:08.313783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200eef30-76b8-42ae-af2f-fd7fcba092fb None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:08.314094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200eef30-76b8-42ae-af2f-fd7fcba092fb None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:08.314369 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200eef30-76b8-42ae-af2f-fd7fcba092fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:08.328226 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200eef30-76b8-42ae-af2f-fd7fcba092fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:08.329979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200eef30-76b8-42ae-af2f-fd7fcba092fb None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-200eef30-76b8-42ae-af2f-fd7fcba092fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:08.350266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:08.351237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:08.352107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:08.352107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:08.352107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:08.352107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:08.356250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 301/602] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:08.357573 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce2a8b-f4f0-49df-87e3-5f4c8dff8278 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:08.357861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce2a8b-f4f0-49df-87e3-5f4c8dff8278 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:08.358167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce2a8b-f4f0-49df-87e3-5f4c8dff8278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:08.358844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:08.382246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-39ce2a8b-f4f0-49df-87e3-5f4c8dff8278 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:08.383810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-39ce2a8b-f4f0-49df-87e3-5f4c8dff8278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:08.396814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704b63d-d04b-49c7-b1cb-13be06cb6688 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:08.437558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08ed54-e215-4aa7-b9eb-b526c5a9d85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:08.438555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08ed54-e215-4aa7-b9eb-b526c5a9d85f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:08.439060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08ed54-e215-4aa7-b9eb-b526c5a9d85f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:08.439572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08ed54-e215-4aa7-b9eb-b526c5a9d85f None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:08.440708 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08ed54-e215-4aa7-b9eb-b526c5a9d85f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:08.441216 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08ed54-e215-4aa7-b9eb-b526c5a9d85f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:08.442955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08ed54-e215-4aa7-b9eb-b526c5a9d85f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:08.454431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:08.457285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 302/603] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:08.458397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08ed54-e215-4aa7-b9eb-b526c5a9d85f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:08.467654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 302/604] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:08.470058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce2a8b-f4f0-49df-87e3-5f4c8dff8278 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:08.602061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08ed54-e215-4aa7-b9eb-b526c5a9d85f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:08.630033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce2a8b-f4f0-49df-87e3-5f4c8dff8278 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:08.640510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f69d9e-0d03-45fe-9818-9557fc2441cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:08.641336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f69d9e-0d03-45fe-9818-9557fc2441cf None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:08.641669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f69d9e-0d03-45fe-9818-9557fc2441cf None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:08.642277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f69d9e-0d03-45fe-9818-9557fc2441cf None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:08.642975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f69d9e-0d03-45fe-9818-9557fc2441cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:08.643290 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f69d9e-0d03-45fe-9818-9557fc2441cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:08.645223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f69d9e-0d03-45fe-9818-9557fc2441cf None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:08.648557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:08.660471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f41261-e934-4039-8a86-41b6439d449f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:08.661151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f41261-e934-4039-8a86-41b6439d449f None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:08.661438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f41261-e934-4039-8a86-41b6439d449f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:08.661811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 303/605] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:08.662825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f69d9e-0d03-45fe-9818-9557fc2441cf None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:08.666340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f41261-e934-4039-8a86-41b6439d449f None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/f3264e17413e4c64b644c1b607cca15b/roles/280f24425d0a4b16951d30c20d7290f9` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:08.683091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f41261-e934-4039-8a86-41b6439d449f None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=f3264e17413e4c64b644c1b607cca15b, role_id=280f24425d0a4b16951d30c20d7290f9)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:08.684646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f41261-e934-4039-8a86-41b6439d449f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:08.776384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f69d9e-0d03-45fe-9818-9557fc2441cf None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:08.794036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 303/606] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:22:08 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/f3264e17413e4c64b644c1b607cca15b/roles/280f24425d0a4b16951d30c20d7290f9 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:22:08.794036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f41261-e934-4039-8a86-41b6439d449f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:08.829686 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:08.841363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bd92b-0cc6-4ab2-a68d-02a34af49ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:08.842974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bd92b-0cc6-4ab2-a68d-02a34af49ca1 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:08.843261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bd92b-0cc6-4ab2-a68d-02a34af49ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:08.843536 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bd92b-0cc6-4ab2-a68d-02a34af49ca1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:08.858675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bd92b-0cc6-4ab2-a68d-02a34af49ca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-873bd92b-0cc6-4ab2-a68d-02a34af49ca1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:08.867301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:08.873484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 304/607] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:08.874372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a016e-7c20-468f-b1d3-16027b339cba None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:08.875133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a016e-7c20-468f-b1d3-16027b339cba None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:08.875133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a016e-7c20-468f-b1d3-16027b339cba None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:08.877104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 305/608] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:08.878126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a016e-7c20-468f-b1d3-16027b339cba None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:08.939920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f41261-e934-4039-8a86-41b6439d449f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:08.958929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:08.998052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0965a-d308-4ace-b347-68bd70ff15a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:08.998052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0965a-d308-4ace-b347-68bd70ff15a8 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:08.998052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0965a-d308-4ace-b347-68bd70ff15a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:08.998052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0965a-d308-4ace-b347-68bd70ff15a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:09.000547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0965a-d308-4ace-b347-68bd70ff15a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:09.002020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0965a-d308-4ace-b347-68bd70ff15a8 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-90b0965a-d308-4ace-b347-68bd70ff15a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:09.014192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:09.015230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:09.016103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:09.016103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:09.016103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:09.016103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:09.016103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:09.016103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:09.016103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:09.016103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:09.016103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 304/609] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:09.017154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d36af-2ceb-4dba-ab5e-9bad33c2ba15 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:09.017154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d36af-2ceb-4dba-ab5e-9bad33c2ba15 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:09.017339 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d36af-2ceb-4dba-ab5e-9bad33c2ba15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:09.028334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a016e-7c20-468f-b1d3-16027b339cba None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:09.036816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-433d36af-2ceb-4dba-ab5e-9bad33c2ba15 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:09.038017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-433d36af-2ceb-4dba-ab5e-9bad33c2ba15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:09.071634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bb2c9-d2bb-4ae2-a466-de9d7090b92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:09.072252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bb2c9-d2bb-4ae2-a466-de9d7090b92a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:09.072641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bb2c9-d2bb-4ae2-a466-de9d7090b92a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:09.072918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bb2c9-d2bb-4ae2-a466-de9d7090b92a None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:09.073689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bb2c9-d2bb-4ae2-a466-de9d7090b92a None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:09.074031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bb2c9-d2bb-4ae2-a466-de9d7090b92a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:09.076526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bb2c9-d2bb-4ae2-a466-de9d7090b92a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:09.089217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 306/610] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:09.090197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bb2c9-d2bb-4ae2-a466-de9d7090b92a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:09.117022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 305/611] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:09.117803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d36af-2ceb-4dba-ab5e-9bad33c2ba15 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:09.146132 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:09.240621 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bb2c9-d2bb-4ae2-a466-de9d7090b92a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:09.244642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d36af-2ceb-4dba-ab5e-9bad33c2ba15 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:09.263524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:09.282978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9c68c-eb78-4e05-9fcd-6fb3b99788fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:09.283780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9c68c-eb78-4e05-9fcd-6fb3b99788fb None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:09.284705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9c68c-eb78-4e05-9fcd-6fb3b99788fb None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:09.285052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9c68c-eb78-4e05-9fcd-6fb3b99788fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:09.285169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298d84c-308c-4a34-a5ee-35cf230e15c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:09.285984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d84c-308c-4a34-a5ee-35cf230e15c7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:09.286362 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d84c-308c-4a34-a5ee-35cf230e15c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:09.286704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d84c-308c-4a34-a5ee-35cf230e15c7 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:09.287819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298d84c-308c-4a34-a5ee-35cf230e15c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:09.288399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298d84c-308c-4a34-a5ee-35cf230e15c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:09.290723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298d84c-308c-4a34-a5ee-35cf230e15c7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:09.298511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9c68c-eb78-4e05-9fcd-6fb3b99788fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:09.301466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9c68c-eb78-4e05-9fcd-6fb3b99788fb None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:09.308265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 307/612] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:09.309374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298d84c-308c-4a34-a5ee-35cf230e15c7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-a3c9c68c-eb78-4e05-9fcd-6fb3b99788fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:09.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:09.324289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:09.325394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:09.325394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:09.325394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:09.325394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:09.330028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 306/613] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:09.330028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9c68c-eb78-4e05-9fcd-6fb3b99788fb None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:09.404418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:09.439444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9c68c-eb78-4e05-9fcd-6fb3b99788fb None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:09.451349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298d84c-308c-4a34-a5ee-35cf230e15c7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:09.477618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebf801-9448-4b99-914f-1c69c11e0030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:09.478340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebf801-9448-4b99-914f-1c69c11e0030 None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:09.478626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebf801-9448-4b99-914f-1c69c11e0030 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:09.478903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebf801-9448-4b99-914f-1c69c11e0030 None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/c98761f81496456fb3b5a5cffd4ce763/roles/280f24425d0a4b16951d30c20d7290f9` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:09.497398 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72917858-c216-4b44-b984-f502addf94c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:09.498435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72917858-c216-4b44-b984-f502addf94c6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:09.498601 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebf801-9448-4b99-914f-1c69c11e0030 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=c98761f81496456fb3b5a5cffd4ce763, role_id=280f24425d0a4b16951d30c20d7290f9)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:09.499008 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72917858-c216-4b44-b984-f502addf94c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:09.499448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72917858-c216-4b44-b984-f502addf94c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:09.500063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebf801-9448-4b99-914f-1c69c11e0030 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:09.511863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72917858-c216-4b44-b984-f502addf94c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-72917858-c216-4b44-b984-f502addf94c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:09.516671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:09.518631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 308/614] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:09.519684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72917858-c216-4b44-b984-f502addf94c6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:09.566840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:09.567986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 307/615] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:22:09 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/c98761f81496456fb3b5a5cffd4ce763/roles/280f24425d0a4b16951d30c20d7290f9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:22:09.569010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebf801-9448-4b99-914f-1c69c11e0030 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:09.642046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72917858-c216-4b44-b984-f502addf94c6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:09.673014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee778e6-f95a-4e1c-946f-5ecf92933ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:09.673599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee778e6-f95a-4e1c-946f-5ecf92933ad0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:09.673894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee778e6-f95a-4e1c-946f-5ecf92933ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:09.674261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee778e6-f95a-4e1c-946f-5ecf92933ad0 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:09.682035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee778e6-f95a-4e1c-946f-5ecf92933ad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:09.682035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee778e6-f95a-4e1c-946f-5ecf92933ad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:09.682035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee778e6-f95a-4e1c-946f-5ecf92933ad0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:09.698029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 309/616] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:09.698029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee778e6-f95a-4e1c-946f-5ecf92933ad0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:09.773670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebf801-9448-4b99-914f-1c69c11e0030 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:09.803158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:09.805316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:09.807549 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee778e6-f95a-4e1c-946f-5ecf92933ad0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:09.845301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ec539-f3a3-4327-983b-391f0ee1d0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:09.846310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ec539-f3a3-4327-983b-391f0ee1d0b6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:09.846831 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ec539-f3a3-4327-983b-391f0ee1d0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:09.847252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ec539-f3a3-4327-983b-391f0ee1d0b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:09.855924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ec539-f3a3-4327-983b-391f0ee1d0b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:09.857697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ec539-f3a3-4327-983b-391f0ee1d0b6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:09.863590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f7929-5a81-4ea0-ba8e-7157f6add568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:09.864233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f7929-5a81-4ea0-ba8e-7157f6add568 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:09.864516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f7929-5a81-4ea0-ba8e-7157f6add568 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:09.864789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f7929-5a81-4ea0-ba8e-7157f6add568 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:09.865519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f7929-5a81-4ea0-ba8e-7157f6add568 None admin] RBAC: Adding query filter params (name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:09.865827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f7929-5a81-4ea0-ba8e-7157f6add568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:09.867303 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f7929-5a81-4ea0-ba8e-7157f6add568 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-737ec539-f3a3-4327-983b-391f0ee1d0b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:09.872490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:09.873623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:09.874669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:09.874669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:09.874669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:09.874669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:09.874669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:09.874669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:09.874669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:09.874669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:09.874669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:09.874669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:09.878153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 310/617] 10.222.0.73 () {56 vars in 1093 bytes} [Thu Aug 4 21:22:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:09.882112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ec539-f3a3-4327-983b-391f0ee1d0b6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:09.886861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 308/618] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:09.892800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f7929-5a81-4ea0-ba8e-7157f6add568 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:10.022829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ec539-f3a3-4327-983b-391f0ee1d0b6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:10.060383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14170038-c44d-40ab-be3c-40d4a513e5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:10.061017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14170038-c44d-40ab-be3c-40d4a513e5cd None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:10.061297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14170038-c44d-40ab-be3c-40d4a513e5cd None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:10.061619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14170038-c44d-40ab-be3c-40d4a513e5cd None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:10.062453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14170038-c44d-40ab-be3c-40d4a513e5cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:10.062757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14170038-c44d-40ab-be3c-40d4a513e5cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:10.064192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14170038-c44d-40ab-be3c-40d4a513e5cd None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:10.076029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f7929-5a81-4ea0-ba8e-7157f6add568 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:10.085249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 311/619] 10.222.0.73 () {56 vars in 1133 bytes} [Thu Aug 4 21:22:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:10.091014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14170038-c44d-40ab-be3c-40d4a513e5cd None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:10.123502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:10.138422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcea752-32b1-4c71-9e15-5e64b931fdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:10.140185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcea752-32b1-4c71-9e15-5e64b931fdce None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:10.140476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcea752-32b1-4c71-9e15-5e64b931fdce None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:10.140755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcea752-32b1-4c71-9e15-5e64b931fdce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:10.158619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcea752-32b1-4c71-9e15-5e64b931fdce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:10.160703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcea752-32b1-4c71-9e15-5e64b931fdce None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-6fcea752-32b1-4c71-9e15-5e64b931fdce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:10.186826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:10.187805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:10.188917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:10.188917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:10.188917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:10.188917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:10.188917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 309/620] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:10.195168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcea752-32b1-4c71-9e15-5e64b931fdce None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:10.253634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:10.286035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14170038-c44d-40ab-be3c-40d4a513e5cd None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:10.342480 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcea752-32b1-4c71-9e15-5e64b931fdce None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:10.352625 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec79e7-4416-4f41-82ae-0e1ce67d0a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:10.353256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec79e7-4416-4f41-82ae-0e1ce67d0a29 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:10.353538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec79e7-4416-4f41-82ae-0e1ce67d0a29 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:10.353812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec79e7-4416-4f41-82ae-0e1ce67d0a29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:10.381904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e969df-921f-4e8b-91db-c829d20a3632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:10.383031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e969df-921f-4e8b-91db-c829d20a3632 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:10.383597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e969df-921f-4e8b-91db-c829d20a3632 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:10.384089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e969df-921f-4e8b-91db-c829d20a3632 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:10.384979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e969df-921f-4e8b-91db-c829d20a3632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:10.385650 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e969df-921f-4e8b-91db-c829d20a3632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:10.387549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e969df-921f-4e8b-91db-c829d20a3632 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:10.401367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 310/621] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:10.403187 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8a23f-a3a2-4292-9e45-9804fb4c48de None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:10.403690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8a23f-a3a2-4292-9e45-9804fb4c48de None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:10.404158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8a23f-a3a2-4292-9e45-9804fb4c48de None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:10.406623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 311/622] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:10.407514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8a23f-a3a2-4292-9e45-9804fb4c48de None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:10.429743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec79e7-4416-4f41-82ae-0e1ce67d0a29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:10.431279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec79e7-4416-4f41-82ae-0e1ce67d0a29 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:10.448489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:10.521138 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:10.535248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8a23f-a3a2-4292-9e45-9804fb4c48de None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:10.538093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 312/623] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:22:10.539740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8523cb-13f7-4ab8-b612-a525eb2ac072 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:10.540231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8523cb-13f7-4ab8-b612-a525eb2ac072 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:10.540701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8523cb-13f7-4ab8-b612-a525eb2ac072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:10.563330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-8b8523cb-13f7-4ab8-b612-a525eb2ac072 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:10.564860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8b8523cb-13f7-4ab8-b612-a525eb2ac072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:10.579290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea8a8-95b0-4501-b290-81331f94ebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:10.579939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea8a8-95b0-4501-b290-81331f94ebfd None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:10.580228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea8a8-95b0-4501-b290-81331f94ebfd None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:10.580599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea8a8-95b0-4501-b290-81331f94ebfd None admin] PATH_INFO: `/v3/users/swift` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:10.589884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fea8a8-95b0-4501-b290-81331f94ebfd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:10.591572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fea8a8-95b0-4501-b290-81331f94ebfd None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-f1fea8a8-95b0-4501-b290-81331f94ebfd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:10.605247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:10.606273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:10.607229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:10.607229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:10.607229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:10.607229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:10.607229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:10.607229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:10.607229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:10.607229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:10.607229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:10.607229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:10.611744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 312/624] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:10.613172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea8a8-95b0-4501-b290-81331f94ebfd None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:10.642424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 313/625] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:10.643821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8523cb-13f7-4ab8-b612-a525eb2ac072 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:10.691669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:10.727534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea8a8-95b0-4501-b290-81331f94ebfd None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:10.748144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8523cb-13f7-4ab8-b612-a525eb2ac072 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:10.766709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e0cd9-87dd-4bc3-bd15-58ff4064847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:10.767560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e0cd9-87dd-4bc3-bd15-58ff4064847c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:10.767993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e0cd9-87dd-4bc3-bd15-58ff4064847c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:10.768516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e0cd9-87dd-4bc3-bd15-58ff4064847c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:10.780655 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e0cd9-87dd-4bc3-bd15-58ff4064847c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:10.782439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e0cd9-87dd-4bc3-bd15-58ff4064847c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:10.783512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9084e-3cd5-401b-8df5-dd9ea11dca95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:10.784774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9084e-3cd5-401b-8df5-dd9ea11dca95 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:10.785146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9084e-3cd5-401b-8df5-dd9ea11dca95 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:10.785507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9084e-3cd5-401b-8df5-dd9ea11dca95 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:10.786655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9084e-3cd5-401b-8df5-dd9ea11dca95 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:10.786776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9084e-3cd5-401b-8df5-dd9ea11dca95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:10.788344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9084e-3cd5-401b-8df5-dd9ea11dca95 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-e54e0cd9-87dd-4bc3-bd15-58ff4064847c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:10.800043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:10.801343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:10.802356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:10.802356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:10.802356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:10.802356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:10.804455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 313/626] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:10.805415 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e0cd9-87dd-4bc3-bd15-58ff4064847c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:10.812452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 314/627] 10.222.0.73 () {56 vars in 1135 bytes} [Thu Aug 4 21:22:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:10.812452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9084e-3cd5-401b-8df5-dd9ea11dca95 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:10.814651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:10.911608 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9084e-3cd5-401b-8df5-dd9ea11dca95 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:10.914426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:10.928233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e0cd9-87dd-4bc3-bd15-58ff4064847c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:10.947412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb2ff6-adae-4d13-b8aa-de1d78047585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:10.948099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb2ff6-adae-4d13-b8aa-de1d78047585 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:10.949513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb2ff6-adae-4d13-b8aa-de1d78047585 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:10.949513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb2ff6-adae-4d13-b8aa-de1d78047585 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:10.949513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb2ff6-adae-4d13-b8aa-de1d78047585 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:10.949730 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb2ff6-adae-4d13-b8aa-de1d78047585 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:10.951158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb2ff6-adae-4d13-b8aa-de1d78047585 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:10.963317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 315/628] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:10.964639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb2ff6-adae-4d13-b8aa-de1d78047585 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:10.967737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0b50f-0a3e-4c47-aaa3-38af73f026b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:10.969613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0b50f-0a3e-4c47-aaa3-38af73f026b5 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:10.970107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0b50f-0a3e-4c47-aaa3-38af73f026b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:10.972628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0b50f-0a3e-4c47-aaa3-38af73f026b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:10.983357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0b50f-0a3e-4c47-aaa3-38af73f026b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-41f0b50f-0a3e-4c47-aaa3-38af73f026b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:10.990041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:10.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:10.991928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:10.994478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 314/629] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:10.995846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0b50f-0a3e-4c47-aaa3-38af73f026b5 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:11.069145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:11.071639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb2ff6-adae-4d13-b8aa-de1d78047585 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:11.109004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0b50f-0a3e-4c47-aaa3-38af73f026b5 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:11.112521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364af5c-2380-4ad1-b306-ed02d7b8aa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:11.113163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364af5c-2380-4ad1-b306-ed02d7b8aa24 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:11.113440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364af5c-2380-4ad1-b306-ed02d7b8aa24 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:11.113709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364af5c-2380-4ad1-b306-ed02d7b8aa24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:11.125505 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364af5c-2380-4ad1-b306-ed02d7b8aa24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:11.127031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364af5c-2380-4ad1-b306-ed02d7b8aa24 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-9364af5c-2380-4ad1-b306-ed02d7b8aa24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:11.141760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:11.142698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:11.144301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:11.144301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:11.144301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:11.144301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:11.144301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 316/630] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:11.144575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364af5c-2380-4ad1-b306-ed02d7b8aa24 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:11.153766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de974a-6f65-4022-acdb-aff127c92beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:11.154716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de974a-6f65-4022-acdb-aff127c92beb None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:11.155262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de974a-6f65-4022-acdb-aff127c92beb None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:11.155715 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de974a-6f65-4022-acdb-aff127c92beb None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:11.163930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de974a-6f65-4022-acdb-aff127c92beb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:11.165567 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:11.166113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de974a-6f65-4022-acdb-aff127c92beb None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-d5de974a-6f65-4022-acdb-aff127c92beb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:11.181941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:11.184716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:11.185814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:11.185814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:11.185814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:11.185814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:11.185814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:11.185814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:11.185814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:11.185814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:11.190217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 315/631] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:11.191993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a10614-3172-4e89-8af5-9e9fd09cffbf None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:11.192348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a10614-3172-4e89-8af5-9e9fd09cffbf None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:11.192625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a10614-3172-4e89-8af5-9e9fd09cffbf None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:11.194312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 316/632] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:11.199127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a10614-3172-4e89-8af5-9e9fd09cffbf None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:11.259963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364af5c-2380-4ad1-b306-ed02d7b8aa24 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:11.297744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fc3a3-f6b0-469b-bfd1-45b029d11234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:11.299383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fc3a3-f6b0-469b-bfd1-45b029d11234 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:11.299901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fc3a3-f6b0-469b-bfd1-45b029d11234 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:11.300314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fc3a3-f6b0-469b-bfd1-45b029d11234 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:11.301166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fc3a3-f6b0-469b-bfd1-45b029d11234 None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:11.301710 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fc3a3-f6b0-469b-bfd1-45b029d11234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:11.303346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fc3a3-f6b0-469b-bfd1-45b029d11234 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:11.308946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a10614-3172-4e89-8af5-9e9fd09cffbf None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:11.316588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 317/633] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:11.317650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fc3a3-f6b0-469b-bfd1-45b029d11234 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:11.333653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:11.344691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2de83d-d8d9-47da-9897-799b54aae135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:11.350025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2de83d-d8d9-47da-9897-799b54aae135 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:11.350025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2de83d-d8d9-47da-9897-799b54aae135 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:11.350025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2de83d-d8d9-47da-9897-799b54aae135 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:11.350025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2de83d-d8d9-47da-9897-799b54aae135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:11.350025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2de83d-d8d9-47da-9897-799b54aae135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:11.350025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2de83d-d8d9-47da-9897-799b54aae135 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:11.361174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 317/634] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:11.362564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e6b74-b34f-4f41-ab81-670c1eda9d9c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:11.362947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e6b74-b34f-4f41-ab81-670c1eda9d9c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:11.363227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e6b74-b34f-4f41-ab81-670c1eda9d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:11.381919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-379e6b74-b34f-4f41-ab81-670c1eda9d9c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:11.383142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-379e6b74-b34f-4f41-ab81-670c1eda9d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:11.405510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:11.429343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fc3a3-f6b0-469b-bfd1-45b029d11234 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:11.462227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 318/635] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:11.463144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e6b74-b34f-4f41-ab81-670c1eda9d9c None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:11.470318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff093f-2f66-469c-814f-5021dde63e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:11.471001 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff093f-2f66-469c-814f-5021dde63e6b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:11.471287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff093f-2f66-469c-814f-5021dde63e6b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:11.471614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff093f-2f66-469c-814f-5021dde63e6b None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:11.472784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff093f-2f66-469c-814f-5021dde63e6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:11.473100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff093f-2f66-469c-814f-5021dde63e6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:11.474767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff093f-2f66-469c-814f-5021dde63e6b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:11.488086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 318/636] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:11.488927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff093f-2f66-469c-814f-5021dde63e6b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:11.588034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e6b74-b34f-4f41-ab81-670c1eda9d9c None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:11.593645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff093f-2f66-469c-814f-5021dde63e6b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:11.603102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:11.624502 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826e33a-e49a-4cea-b013-d277724f00e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:11.625173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826e33a-e49a-4cea-b013-d277724f00e1 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:11.625457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826e33a-e49a-4cea-b013-d277724f00e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:11.625734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826e33a-e49a-4cea-b013-d277724f00e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:11.630881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a5b8b-aa1d-410a-89d3-ca693dbe62f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:11.631738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a5b8b-aa1d-410a-89d3-ca693dbe62f9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:11.632197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a5b8b-aa1d-410a-89d3-ca693dbe62f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:11.632663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a5b8b-aa1d-410a-89d3-ca693dbe62f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:11.636787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826e33a-e49a-4cea-b013-d277724f00e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:11.638803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826e33a-e49a-4cea-b013-d277724f00e1 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:11.641115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a5b8b-aa1d-410a-89d3-ca693dbe62f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:11.643097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a5b8b-aa1d-410a-89d3-ca693dbe62f9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:11.646239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-0826e33a-e49a-4cea-b013-d277724f00e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:11.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:11.662366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:11.663276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:11.663276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:11.663276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:11.663276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:11.663276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 319/637] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:11.663276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826e33a-e49a-4cea-b013-d277724f00e1 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-ac9a5b8b-aa1d-410a-89d3-ca693dbe62f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:11.663645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:11.664559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:11.665439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:11.665439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:11.665439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:11.665439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:11.665439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:11.665439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:11.665439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:11.665439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:11.665439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 319/638] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:11.665439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a5b8b-aa1d-410a-89d3-ca693dbe62f9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:11.770865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a5b8b-aa1d-410a-89d3-ca693dbe62f9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:11.773546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826e33a-e49a-4cea-b013-d277724f00e1 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:11.806199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79900a09-8a58-407b-8458-2dc1c38355d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:11.806819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79900a09-8a58-407b-8458-2dc1c38355d1 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:11.807130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79900a09-8a58-407b-8458-2dc1c38355d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:11.807410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79900a09-8a58-407b-8458-2dc1c38355d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:11.808197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79900a09-8a58-407b-8458-2dc1c38355d1 None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=e7b8267780364a2181abf5817b54ce5c) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:11.808514 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79900a09-8a58-407b-8458-2dc1c38355d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:11.809625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30225731-e9ac-466e-8d9d-436e1ba26d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:11.810145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79900a09-8a58-407b-8458-2dc1c38355d1 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:11.810619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30225731-e9ac-466e-8d9d-436e1ba26d63 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:11.811040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30225731-e9ac-466e-8d9d-436e1ba26d63 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:11.811455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30225731-e9ac-466e-8d9d-436e1ba26d63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:11.817095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 320/639] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:11 2022] GET /identity/v3/role_assignments?user.id=e7b8267780364a2181abf5817b54ce5c&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:11.817896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79900a09-8a58-407b-8458-2dc1c38355d1 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:11.833841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30225731-e9ac-466e-8d9d-436e1ba26d63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-30225731-e9ac-466e-8d9d-436e1ba26d63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:11.840025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:11.841285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:11.842207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 320/640] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:11.842933 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30225731-e9ac-466e-8d9d-436e1ba26d63 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:11.879317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:11.900098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:11.976842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30225731-e9ac-466e-8d9d-436e1ba26d63 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:12.019294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba51ea5-33ba-437f-bea3-9dd8e31ed2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:12.020116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51ea5-33ba-437f-bea3-9dd8e31ed2ca None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.020393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51ea5-33ba-437f-bea3-9dd8e31ed2ca None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.020665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51ea5-33ba-437f-bea3-9dd8e31ed2ca None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:12.021426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba51ea5-33ba-437f-bea3-9dd8e31ed2ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:12.021730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba51ea5-33ba-437f-bea3-9dd8e31ed2ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:12.023230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba51ea5-33ba-437f-bea3-9dd8e31ed2ca None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:12.038690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 321/641] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:12.044171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba51ea5-33ba-437f-bea3-9dd8e31ed2ca None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:12.073623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79900a09-8a58-407b-8458-2dc1c38355d1 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:12.113235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e3e4f-c8b6-4949-92a8-898e3d65a3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:12.113899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e3e4f-c8b6-4949-92a8-898e3d65a3b9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.114211 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e3e4f-c8b6-4949-92a8-898e3d65a3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.114490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e3e4f-c8b6-4949-92a8-898e3d65a3b9 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:12.115182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e3e4f-c8b6-4949-92a8-898e3d65a3b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:12.115489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e3e4f-c8b6-4949-92a8-898e3d65a3b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:12.116853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e3e4f-c8b6-4949-92a8-898e3d65a3b9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:12.128886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 321/642] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:12.129951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d30d2-338f-4875-8a7a-e39a0bbf25b3 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.130276 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d30d2-338f-4875-8a7a-e39a0bbf25b3 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.130550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d30d2-338f-4875-8a7a-e39a0bbf25b3 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:12.131852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 322/643] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:12.132397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d30d2-338f-4875-8a7a-e39a0bbf25b3 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:12.149155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:12.165025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba51ea5-33ba-437f-bea3-9dd8e31ed2ca None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:12.202236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0803a5-c408-4999-a01e-4a9178e50f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:12.203162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0803a5-c408-4999-a01e-4a9178e50f04 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.203639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0803a5-c408-4999-a01e-4a9178e50f04 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.203910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0803a5-c408-4999-a01e-4a9178e50f04 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:12.204628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0803a5-c408-4999-a01e-4a9178e50f04 None admin] RBAC: Adding query filter params (name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:12.204933 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0803a5-c408-4999-a01e-4a9178e50f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:12.206489 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0803a5-c408-4999-a01e-4a9178e50f04 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:12.214897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 322/644] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:12.215940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0803a5-c408-4999-a01e-4a9178e50f04 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:12.247507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d30d2-338f-4875-8a7a-e39a0bbf25b3 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:12.271304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:12.283055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0492b57-4f95-4299-bf58-1b53278b79c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:12.283726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0492b57-4f95-4299-bf58-1b53278b79c4 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.284608 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0492b57-4f95-4299-bf58-1b53278b79c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.285116 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0492b57-4f95-4299-bf58-1b53278b79c4 None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/3342e1039131496ba121b04685ec7b3d/roles/280f24425d0a4b16951d30c20d7290f9` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:12.299358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0492b57-4f95-4299-bf58-1b53278b79c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=3342e1039131496ba121b04685ec7b3d, role_id=280f24425d0a4b16951d30c20d7290f9)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:12.301724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0492b57-4f95-4299-bf58-1b53278b79c4 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:12.325121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0803a5-c408-4999-a01e-4a9178e50f04 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:12.359379 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd702689-135a-4757-b4ee-fd067210812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:12.360055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd702689-135a-4757-b4ee-fd067210812d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.360299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd702689-135a-4757-b4ee-fd067210812d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.360575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd702689-135a-4757-b4ee-fd067210812d None admin] PATH_INFO: `/v3/users/glance` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:12.369500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 323/645] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:22:12 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/3342e1039131496ba121b04685ec7b3d/roles/280f24425d0a4b16951d30c20d7290f9 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:22:12.370719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e694c43-fa6f-49e9-971d-ccfee96c85fc None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.370958 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd702689-135a-4757-b4ee-fd067210812d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:12.371198 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e694c43-fa6f-49e9-971d-ccfee96c85fc None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.371572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e694c43-fa6f-49e9-971d-ccfee96c85fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:12.373177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd702689-135a-4757-b4ee-fd067210812d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-dd702689-135a-4757-b4ee-fd067210812d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:12.388873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:12.389951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:12.391271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:12.391271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:12.391271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:12.391271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:12.391271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:12.391271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:12.391271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:12.391271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:12.391271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 21:22:12.391271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:12.399408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 323/646] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:22:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:12.399408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd702689-135a-4757-b4ee-fd067210812d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:12.409888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-7e694c43-fa6f-49e9-971d-ccfee96c85fc None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:12.411504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7e694c43-fa6f-49e9-971d-ccfee96c85fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:12.413030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:12.494857 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 324/647] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:12.496238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e694c43-fa6f-49e9-971d-ccfee96c85fc None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:12.515053 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd702689-135a-4757-b4ee-fd067210812d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:12.554581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbfda5-f85e-44c7-9426-c0a4e738d9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:12.555180 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbfda5-f85e-44c7-9426-c0a4e738d9ea None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.555516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbfda5-f85e-44c7-9426-c0a4e738d9ea None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.555796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbfda5-f85e-44c7-9426-c0a4e738d9ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:12.568660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbfda5-f85e-44c7-9426-c0a4e738d9ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:12.570268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbfda5-f85e-44c7-9426-c0a4e738d9ea None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-98bbfda5-f85e-44c7-9426-c0a4e738d9ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:12.585784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:12.586927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:12.588693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:12.588693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:12.588693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:12.588693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:12.588693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 324/648] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:12.588693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbfda5-f85e-44c7-9426-c0a4e738d9ea None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:12.636787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e694c43-fa6f-49e9-971d-ccfee96c85fc None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:12.638716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:12.660798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:12.684440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92649ab3-780b-4e37-ab57-469cd2c2f4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:12.685316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92649ab3-780b-4e37-ab57-469cd2c2f4a1 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.685746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92649ab3-780b-4e37-ab57-469cd2c2f4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.686555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92649ab3-780b-4e37-ab57-469cd2c2f4a1 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:12.687588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92649ab3-780b-4e37-ab57-469cd2c2f4a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:12.688387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92649ab3-780b-4e37-ab57-469cd2c2f4a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:12.690900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92649ab3-780b-4e37-ab57-469cd2c2f4a1 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:12.712439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbfda5-f85e-44c7-9426-c0a4e738d9ea None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:12.717173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 325/649] 10.222.0.73 () {56 vars in 1137 bytes} [Thu Aug 4 21:22:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:12.723200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92649ab3-780b-4e37-ab57-469cd2c2f4a1 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:12.760414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a25ed-b7d5-4d36-8a20-32417c3e8a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:12.761340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a25ed-b7d5-4d36-8a20-32417c3e8a7b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.761927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a25ed-b7d5-4d36-8a20-32417c3e8a7b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.762376 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a25ed-b7d5-4d36-8a20-32417c3e8a7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:12.776888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a25ed-b7d5-4d36-8a20-32417c3e8a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-703a25ed-b7d5-4d36-8a20-32417c3e8a7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:12.783986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:12.785104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:12.786170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:12.789112 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 325/650] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:12.790079 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a25ed-b7d5-4d36-8a20-32417c3e8a7b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:12.840087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92649ab3-780b-4e37-ab57-469cd2c2f4a1 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:12.878307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda18b5-a4ea-4475-b5e6-b3d5f408b9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:12.879121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda18b5-a4ea-4475-b5e6-b3d5f408b9ac None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.879489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda18b5-a4ea-4475-b5e6-b3d5f408b9ac None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.879865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda18b5-a4ea-4475-b5e6-b3d5f408b9ac None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:12.880658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda18b5-a4ea-4475-b5e6-b3d5f408b9ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:12.881090 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda18b5-a4ea-4475-b5e6-b3d5f408b9ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:12.882582 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda18b5-a4ea-4475-b5e6-b3d5f408b9ac None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:12.896174 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 326/651] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:12.897159 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda18b5-a4ea-4475-b5e6-b3d5f408b9ac None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:12.923903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:12.928391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a25ed-b7d5-4d36-8a20-32417c3e8a7b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:12.956199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:12.977953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5252e-203a-4414-9102-273764dc70ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:12.978838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5252e-203a-4414-9102-273764dc70ec None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:12.979269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5252e-203a-4414-9102-273764dc70ec None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:12.979734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5252e-203a-4414-9102-273764dc70ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:13.009353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda18b5-a4ea-4475-b5e6-b3d5f408b9ac None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:13.052430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11441959-8916-455e-a8e1-b3af5078ce7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:13.053066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11441959-8916-455e-a8e1-b3af5078ce7d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:13.053348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11441959-8916-455e-a8e1-b3af5078ce7d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:13.053628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11441959-8916-455e-a8e1-b3af5078ce7d None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:13.054488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11441959-8916-455e-a8e1-b3af5078ce7d None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:13.054796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11441959-8916-455e-a8e1-b3af5078ce7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:13.056425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11441959-8916-455e-a8e1-b3af5078ce7d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:13.065695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 327/652] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:13.066579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11441959-8916-455e-a8e1-b3af5078ce7d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:13.076713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5252e-203a-4414-9102-273764dc70ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:13.078927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5252e-203a-4414-9102-273764dc70ec None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:13.169407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:13.187881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11441959-8916-455e-a8e1-b3af5078ce7d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:13.215407 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 326/653] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:22:13.215407 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5252e-203a-4414-9102-273764dc70ec None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:13.235528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c49747-b40b-4408-b2a9-1979881aa515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:13.236643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c49747-b40b-4408-b2a9-1979881aa515 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:13.237102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c49747-b40b-4408-b2a9-1979881aa515 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:13.237546 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c49747-b40b-4408-b2a9-1979881aa515 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:13.238638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:13.249209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c49747-b40b-4408-b2a9-1979881aa515 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:13.250868 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c49747-b40b-4408-b2a9-1979881aa515 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-10c49747-b40b-4408-b2a9-1979881aa515 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:13.266474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:13.267572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:13.269104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:13.269104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:13.269104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:13.269104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:13.269104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:13.269104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:13.269104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:13.269104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:13.269104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:22:13.269104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:13.277634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 328/654] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:22:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:13.277634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c49747-b40b-4408-b2a9-1979881aa515 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:13.343274 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5252e-203a-4414-9102-273764dc70ec None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:13.378882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b97a0-9741-4557-b1a6-bffc66584117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:13.379668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b97a0-9741-4557-b1a6-bffc66584117 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:13.380089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b97a0-9741-4557-b1a6-bffc66584117 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:13.380406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b97a0-9741-4557-b1a6-bffc66584117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:13.391158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b97a0-9741-4557-b1a6-bffc66584117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:13.392720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b97a0-9741-4557-b1a6-bffc66584117 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:13.399479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c49747-b40b-4408-b2a9-1979881aa515 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-d32b97a0-9741-4557-b1a6-bffc66584117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:13.409826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:13.410973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:13.412090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:13.412090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:13.412090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:13.412090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:13.413741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 327/655] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:13.414681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b97a0-9741-4557-b1a6-bffc66584117 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:13.450424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a37e6d-4e3e-4171-b533-faf5f9e77088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:13.452229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a37e6d-4e3e-4171-b533-faf5f9e77088 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:13.452666 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a37e6d-4e3e-4171-b533-faf5f9e77088 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:13.453091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a37e6d-4e3e-4171-b533-faf5f9e77088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:13.468487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a37e6d-4e3e-4171-b533-faf5f9e77088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:13.470613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a37e6d-4e3e-4171-b533-faf5f9e77088 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:13.473810 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:13.492955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-94a37e6d-4e3e-4171-b533-faf5f9e77088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:13.494636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:13.495706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:13.496824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:13.496824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:13.496824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:13.496824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:13.498352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 329/656] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:13.503951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a37e6d-4e3e-4171-b533-faf5f9e77088 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:13.544070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b97a0-9741-4557-b1a6-bffc66584117 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:13.584674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075bfab-e3d4-4ea0-b0c0-34df2f85215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:13.585292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075bfab-e3d4-4ea0-b0c0-34df2f85215a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:13.585865 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075bfab-e3d4-4ea0-b0c0-34df2f85215a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:13.586192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075bfab-e3d4-4ea0-b0c0-34df2f85215a None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:13.586993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075bfab-e3d4-4ea0-b0c0-34df2f85215a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:13.587303 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075bfab-e3d4-4ea0-b0c0-34df2f85215a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:13.588729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075bfab-e3d4-4ea0-b0c0-34df2f85215a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:13.618085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 328/657] 10.222.0.73 () {56 vars in 1137 bytes} [Thu Aug 4 21:22:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:13.618085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075bfab-e3d4-4ea0-b0c0-34df2f85215a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:13.656755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a37e6d-4e3e-4171-b533-faf5f9e77088 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:13.699233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6716ec-6678-4466-ae4a-9f9e4063ad23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:13.699487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6716ec-6678-4466-ae4a-9f9e4063ad23 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:13.699740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6716ec-6678-4466-ae4a-9f9e4063ad23 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:13.700014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6716ec-6678-4466-ae4a-9f9e4063ad23 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:13.700762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6716ec-6678-4466-ae4a-9f9e4063ad23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:13.701007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6716ec-6678-4466-ae4a-9f9e4063ad23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:13.702385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6716ec-6678-4466-ae4a-9f9e4063ad23 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:13.716554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 330/658] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:13.717509 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6716ec-6678-4466-ae4a-9f9e4063ad23 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:13.733019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075bfab-e3d4-4ea0-b0c0-34df2f85215a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:13.764977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:13.771905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813b3cd-1294-4a2a-8608-da53581c5633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:13.772942 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813b3cd-1294-4a2a-8608-da53581c5633 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:13.773385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813b3cd-1294-4a2a-8608-da53581c5633 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:13.773806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813b3cd-1294-4a2a-8608-da53581c5633 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:13.774691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813b3cd-1294-4a2a-8608-da53581c5633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:13.775146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813b3cd-1294-4a2a-8608-da53581c5633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:13.776785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813b3cd-1294-4a2a-8608-da53581c5633 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:13.778413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:13.795306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 329/659] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:13.796493 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813b3cd-1294-4a2a-8608-da53581c5633 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:13.828265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6716ec-6678-4466-ae4a-9f9e4063ad23 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:13.867286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365bbeb-8be9-41d4-8eda-5ce7c79fd90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:13.867966 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365bbeb-8be9-41d4-8eda-5ce7c79fd90e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:13.868255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365bbeb-8be9-41d4-8eda-5ce7c79fd90e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:13.868524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365bbeb-8be9-41d4-8eda-5ce7c79fd90e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:13.908182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813b3cd-1294-4a2a-8608-da53581c5633 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:13.943411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365bbeb-8be9-41d4-8eda-5ce7c79fd90e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:13.945123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365bbeb-8be9-41d4-8eda-5ce7c79fd90e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:13.946350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbd135-817c-4d22-9485-70832057327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:13.947261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbd135-817c-4d22-9485-70832057327d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:13.947722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbd135-817c-4d22-9485-70832057327d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:13.948249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbd135-817c-4d22-9485-70832057327d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:13.962502 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbd135-817c-4d22-9485-70832057327d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:13.962502 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbd135-817c-4d22-9485-70832057327d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-96fbd135-817c-4d22-9485-70832057327d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:13.971982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:13.973465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:13.974527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:13.974527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:13.974527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:13.974527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:13.974527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:13.974527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:13.974527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:13.974527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:13.974527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 21:22:13.974527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:13.976830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 330/660] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:13.977931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbd135-817c-4d22-9485-70832057327d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:14.004327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:14.035204 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:14.053602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 331/661] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:22:14.054404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365bbeb-8be9-41d4-8eda-5ce7c79fd90e None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:14.089387 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbd135-817c-4d22-9485-70832057327d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:14.126984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9eb7f-dc23-4355-a983-b119e362e923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:14.127657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9eb7f-dc23-4355-a983-b119e362e923 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.127949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9eb7f-dc23-4355-a983-b119e362e923 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.128234 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9eb7f-dc23-4355-a983-b119e362e923 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.135137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9eb7f-dc23-4355-a983-b119e362e923 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:14.136589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9eb7f-dc23-4355-a983-b119e362e923 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-60f9eb7f-dc23-4355-a983-b119e362e923 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:14.148886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:14.149932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:14.150996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:14.150996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:14.150996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:14.150996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:14.150996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:14.150996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:14.150996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:14.150996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:14.158739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 331/662] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:14.160593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77907-5945-4ea5-93c4-c6617bb2c0e5 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.161325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77907-5945-4ea5-93c4-c6617bb2c0e5 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.162371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77907-5945-4ea5-93c4-c6617bb2c0e5 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.164502 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 332/663] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:14.165159 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af77907-5945-4ea5-93c4-c6617bb2c0e5 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:14.172525 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365bbeb-8be9-41d4-8eda-5ce7c79fd90e None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:14.215453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792a7b8-921d-45ea-9038-a299f662b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:14.215986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792a7b8-921d-45ea-9038-a299f662b2a6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.216267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792a7b8-921d-45ea-9038-a299f662b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.216542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792a7b8-921d-45ea-9038-a299f662b2a6 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.217329 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792a7b8-921d-45ea-9038-a299f662b2a6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:14.217639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792a7b8-921d-45ea-9038-a299f662b2a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:14.219097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792a7b8-921d-45ea-9038-a299f662b2a6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:14.238896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 332/664] 10.222.0.73 () {56 vars in 1139 bytes} [Thu Aug 4 21:22:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:14.239639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792a7b8-921d-45ea-9038-a299f662b2a6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:14.255293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:14.287006 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af77907-5945-4ea5-93c4-c6617bb2c0e5 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:14.297476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:14.321676 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d703e-3bf1-48d8-b8ca-c6e94a6918fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:14.322348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d703e-3bf1-48d8-b8ca-c6e94a6918fd None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.322634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d703e-3bf1-48d8-b8ca-c6e94a6918fd None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.322910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d703e-3bf1-48d8-b8ca-c6e94a6918fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.334496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d703e-3bf1-48d8-b8ca-c6e94a6918fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:14.335955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d703e-3bf1-48d8-b8ca-c6e94a6918fd None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-1f3d703e-3bf1-48d8-b8ca-c6e94a6918fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:14.352184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:14.353129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:14.354140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:14.354140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:14.354140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:14.354140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:14.354140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 333/665] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:14.355066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab981fc7-4c3e-433c-814d-8ada392be151 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.355424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab981fc7-4c3e-433c-814d-8ada392be151 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.355772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab981fc7-4c3e-433c-814d-8ada392be151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.360341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792a7b8-921d-45ea-9038-a299f662b2a6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:14.377169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ab981fc7-4c3e-433c-814d-8ada392be151 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:14.378950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ab981fc7-4c3e-433c-814d-8ada392be151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:14.398201 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace6573-1e85-4d45-8aea-cc9afcdfdf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:14.398821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6573-1e85-4d45-8aea-cc9afcdfdf5f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.399105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6573-1e85-4d45-8aea-cc9afcdfdf5f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.399379 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6573-1e85-4d45-8aea-cc9afcdfdf5f None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.400125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace6573-1e85-4d45-8aea-cc9afcdfdf5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:14.400637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace6573-1e85-4d45-8aea-cc9afcdfdf5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:14.401828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace6573-1e85-4d45-8aea-cc9afcdfdf5f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:14.422298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 333/666] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:14.422298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace6573-1e85-4d45-8aea-cc9afcdfdf5f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:14.454059 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 334/667] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:14.454253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab981fc7-4c3e-433c-814d-8ada392be151 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:14.504603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:14.533821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace6573-1e85-4d45-8aea-cc9afcdfdf5f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:14.551183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:14.559435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab981fc7-4c3e-433c-814d-8ada392be151 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:14.569124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf5e92-8615-4392-bbf7-79f8d0ba5bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:14.569747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5e92-8615-4392-bbf7-79f8d0ba5bfd None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.570047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5e92-8615-4392-bbf7-79f8d0ba5bfd None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.570326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5e92-8615-4392-bbf7-79f8d0ba5bfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.593824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28afa6-ded3-4acd-b8cc-c6fffea2c8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:14.594465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28afa6-ded3-4acd-b8cc-c6fffea2c8b9 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.594742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28afa6-ded3-4acd-b8cc-c6fffea2c8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.595018 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28afa6-ded3-4acd-b8cc-c6fffea2c8b9 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.595694 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28afa6-ded3-4acd-b8cc-c6fffea2c8b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:14.595996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28afa6-ded3-4acd-b8cc-c6fffea2c8b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:14.597594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28afa6-ded3-4acd-b8cc-c6fffea2c8b9 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:14.608471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 335/668] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:14.609250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28afa6-ded3-4acd-b8cc-c6fffea2c8b9 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:14.640184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf5e92-8615-4392-bbf7-79f8d0ba5bfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:14.641614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf5e92-8615-4392-bbf7-79f8d0ba5bfd None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:14.709176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28afa6-ded3-4acd-b8cc-c6fffea2c8b9 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:14.709721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:14.740167 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 334/669] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:22:14.740167 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c6033-034a-4fc8-90ce-f886dd95135b None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.740167 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c6033-034a-4fc8-90ce-f886dd95135b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.740167 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c6033-034a-4fc8-90ce-f886dd95135b None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.745002 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb342e-1d38-4f8e-9de4-edc32b404711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:14.745134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 335/670] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:14.745229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb342e-1d38-4f8e-9de4-edc32b404711 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.745229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb342e-1d38-4f8e-9de4-edc32b404711 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.745379 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c6033-034a-4fc8-90ce-f886dd95135b None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:14.745468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb342e-1d38-4f8e-9de4-edc32b404711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.745468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb342e-1d38-4f8e-9de4-edc32b404711 None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=03a9a6bcfa1b469298774e09a8912b93) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:14.745634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb342e-1d38-4f8e-9de4-edc32b404711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:14.746962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb342e-1d38-4f8e-9de4-edc32b404711 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:14.753279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 336/671] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:14 2022] GET /identity/v3/role_assignments?user.id=03a9a6bcfa1b469298774e09a8912b93&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:14.754091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb342e-1d38-4f8e-9de4-edc32b404711 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:14.774400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:14.869486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb342e-1d38-4f8e-9de4-edc32b404711 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:14.875438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c6033-034a-4fc8-90ce-f886dd95135b None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:14.908391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78fb18-3370-4409-9bfe-02a2dc0245ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:14.909743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78fb18-3370-4409-9bfe-02a2dc0245ba None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.910029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78fb18-3370-4409-9bfe-02a2dc0245ba None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.910351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78fb18-3370-4409-9bfe-02a2dc0245ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.917651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad18d90-a2a2-4acb-b940-1264eb412a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:14.918344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78fb18-3370-4409-9bfe-02a2dc0245ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:14.918478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18d90-a2a2-4acb-b940-1264eb412a9a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.918630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18d90-a2a2-4acb-b940-1264eb412a9a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.918902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18d90-a2a2-4acb-b940-1264eb412a9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.919797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78fb18-3370-4409-9bfe-02a2dc0245ba None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-7d78fb18-3370-4409-9bfe-02a2dc0245ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:14.931978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:14.933543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:14.934901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:14.934901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:14.934901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:14.934901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:14.934901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:14.934901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:14.934901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:14.934901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:14.935322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad18d90-a2a2-4acb-b940-1264eb412a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:14.935395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 337/672] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:14.935395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78fb18-3370-4409-9bfe-02a2dc0245ba None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-8ad18d90-a2a2-4acb-b940-1264eb412a9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:14.939454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:14.941034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:14.941931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 336/673] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:14.943770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92b38a-eea1-4c8b-bc1a-14258c828ffa None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:14.943770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92b38a-eea1-4c8b-bc1a-14258c828ffa None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:14.943770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92b38a-eea1-4c8b-bc1a-14258c828ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:14.963931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-3f92b38a-eea1-4c8b-bc1a-14258c828ffa None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:14.964870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3f92b38a-eea1-4c8b-bc1a-14258c828ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:14.995686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:15.020398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:15.044299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78fb18-3370-4409-9bfe-02a2dc0245ba None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:15.046316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 337/674] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:15.047396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92b38a-eea1-4c8b-bc1a-14258c828ffa None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:15.081925 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e9b18-854c-405f-8297-382e7c7e8785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:15.082745 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e9b18-854c-405f-8297-382e7c7e8785 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:15.083345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e9b18-854c-405f-8297-382e7c7e8785 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:15.083658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e9b18-854c-405f-8297-382e7c7e8785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:15.095653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e9b18-854c-405f-8297-382e7c7e8785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:15.097600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e9b18-854c-405f-8297-382e7c7e8785 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-562e9b18-854c-405f-8297-382e7c7e8785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:15.113363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:15.114653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:15.115772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:15.115772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:15.115772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:15.115772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:15.119143 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 338/675] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:15.119765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e9b18-854c-405f-8297-382e7c7e8785 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:15.157669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92b38a-eea1-4c8b-bc1a-14258c828ffa None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:15.197767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4703e86-8ce5-4d0c-be09-854b3abb0459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:15.198325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4703e86-8ce5-4d0c-be09-854b3abb0459 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:15.198602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4703e86-8ce5-4d0c-be09-854b3abb0459 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:15.198876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4703e86-8ce5-4d0c-be09-854b3abb0459 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:15.199617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4703e86-8ce5-4d0c-be09-854b3abb0459 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:15.199924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4703e86-8ce5-4d0c-be09-854b3abb0459 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:15.201351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4703e86-8ce5-4d0c-be09-854b3abb0459 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:15.220358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 338/676] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:15.220358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4703e86-8ce5-4d0c-be09-854b3abb0459 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:15.229260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e9b18-854c-405f-8297-382e7c7e8785 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:15.248909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:15.258464 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:15.268069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53526a8a-ec29-4ce4-8368-37982a60cafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:15.268997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53526a8a-ec29-4ce4-8368-37982a60cafa None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:15.269662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53526a8a-ec29-4ce4-8368-37982a60cafa None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:15.270323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53526a8a-ec29-4ce4-8368-37982a60cafa None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:15.271147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53526a8a-ec29-4ce4-8368-37982a60cafa None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:15.271703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53526a8a-ec29-4ce4-8368-37982a60cafa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:15.273800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53526a8a-ec29-4ce4-8368-37982a60cafa None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:15.282976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 339/677] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:22:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:15.283785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53526a8a-ec29-4ce4-8368-37982a60cafa None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:15.335714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4703e86-8ce5-4d0c-be09-854b3abb0459 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:15.371609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d81c8-a65b-412b-b3c7-2971f558d95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:15.372248 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d81c8-a65b-412b-b3c7-2971f558d95f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:15.372526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d81c8-a65b-412b-b3c7-2971f558d95f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:15.372797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d81c8-a65b-412b-b3c7-2971f558d95f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:15.383608 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d81c8-a65b-412b-b3c7-2971f558d95f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-e43d81c8-a65b-412b-b3c7-2971f558d95f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:15.388487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:15.389440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:15.390353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 339/678] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:15.391074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d81c8-a65b-412b-b3c7-2971f558d95f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:15.402972 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53526a8a-ec29-4ce4-8368-37982a60cafa None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:15.446020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f6098-4e7f-4676-ac58-2396e70352b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:15.447275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f6098-4e7f-4676-ac58-2396e70352b3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:15.447891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f6098-4e7f-4676-ac58-2396e70352b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:15.448701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f6098-4e7f-4676-ac58-2396e70352b3 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:15.449844 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f6098-4e7f-4676-ac58-2396e70352b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:15.450497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f6098-4e7f-4676-ac58-2396e70352b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:15.452651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f6098-4e7f-4676-ac58-2396e70352b3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:15.472998 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 340/679] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:15.472998 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f6098-4e7f-4676-ac58-2396e70352b3 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:15.479799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:15.499523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d81c8-a65b-412b-b3c7-2971f558d95f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:15.533011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:15.588731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82945f86-830f-4bb6-acd3-9b817c7d40db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:15.588731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82945f86-830f-4bb6-acd3-9b817c7d40db None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:15.588731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82945f86-830f-4bb6-acd3-9b817c7d40db None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:15.588731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82945f86-830f-4bb6-acd3-9b817c7d40db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:15.589792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82945f86-830f-4bb6-acd3-9b817c7d40db None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=f3264e17413e4c64b644c1b607cca15b) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:15.589792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82945f86-830f-4bb6-acd3-9b817c7d40db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:15.589792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82945f86-830f-4bb6-acd3-9b817c7d40db None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:15.589792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 340/680] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:15 2022] GET /identity/v3/role_assignments?user.id=f3264e17413e4c64b644c1b607cca15b&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:15.589792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82945f86-830f-4bb6-acd3-9b817c7d40db None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:15.590249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f6098-4e7f-4676-ac58-2396e70352b3 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:15.626483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25d5b3-b7d9-4295-96e7-ad7de619bca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:15.627812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25d5b3-b7d9-4295-96e7-ad7de619bca5 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:15.628924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25d5b3-b7d9-4295-96e7-ad7de619bca5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:15.629474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25d5b3-b7d9-4295-96e7-ad7de619bca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:15.642553 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25d5b3-b7d9-4295-96e7-ad7de619bca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:15.644763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25d5b3-b7d9-4295-96e7-ad7de619bca5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-ea25d5b3-b7d9-4295-96e7-ad7de619bca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:15.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:15.663098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:15.664262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:15.664262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:15.664262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:15.664262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:15.666117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 341/681] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:15.667215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25d5b3-b7d9-4295-96e7-ad7de619bca5 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:15.698452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82945f86-830f-4bb6-acd3-9b817c7d40db None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:15.747065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b450fb9-4b33-4650-b0dd-f30f37689d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:15.747876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b450fb9-4b33-4650-b0dd-f30f37689d5a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:15.748263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b450fb9-4b33-4650-b0dd-f30f37689d5a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:15.749369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b450fb9-4b33-4650-b0dd-f30f37689d5a None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:15.750218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b450fb9-4b33-4650-b0dd-f30f37689d5a None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:15.750624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b450fb9-4b33-4650-b0dd-f30f37689d5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:15.752105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b450fb9-4b33-4650-b0dd-f30f37689d5a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:15.761095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 341/682] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:15.762028 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b450fb9-4b33-4650-b0dd-f30f37689d5a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:15.774886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:15.786583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25d5b3-b7d9-4295-96e7-ad7de619bca5 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:15.835959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38089141-3d42-403f-9455-132fc8c69de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:15.836878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38089141-3d42-403f-9455-132fc8c69de1 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:15.837224 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38089141-3d42-403f-9455-132fc8c69de1 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:15.837815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38089141-3d42-403f-9455-132fc8c69de1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:15.847527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38089141-3d42-403f-9455-132fc8c69de1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:15.850152 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38089141-3d42-403f-9455-132fc8c69de1 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-38089141-3d42-403f-9455-132fc8c69de1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:15.867703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:15.868883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:15.870101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:15.870101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:15.870101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:15.870101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:15.870101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:15.870101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:15.870101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:15.870101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:15.870101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 342/683] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:15.874124 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38089141-3d42-403f-9455-132fc8c69de1 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:15.876955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:15.897373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b450fb9-4b33-4650-b0dd-f30f37689d5a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:15.945058 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c74c71-e124-440b-b64f-e295ce0aabf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:15.946252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74c71-e124-440b-b64f-e295ce0aabf4 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:15.946954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74c71-e124-440b-b64f-e295ce0aabf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:15.947406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74c71-e124-440b-b64f-e295ce0aabf4 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:15.948945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c74c71-e124-440b-b64f-e295ce0aabf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:15.949413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c74c71-e124-440b-b64f-e295ce0aabf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:15.951878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c74c71-e124-440b-b64f-e295ce0aabf4 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:15.969172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 342/684] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:15.970364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c74c71-e124-440b-b64f-e295ce0aabf4 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:16.057219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:16.096684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c74c71-e124-440b-b64f-e295ce0aabf4 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:16.123022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38089141-3d42-403f-9455-132fc8c69de1 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:16.132917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15456066-634e-4f8c-95a9-885dd350db4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:16.133907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15456066-634e-4f8c-95a9-885dd350db4b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:16.133907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15456066-634e-4f8c-95a9-885dd350db4b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:16.134158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15456066-634e-4f8c-95a9-885dd350db4b None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:16.134882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15456066-634e-4f8c-95a9-885dd350db4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:16.135187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15456066-634e-4f8c-95a9-885dd350db4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:16.136723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15456066-634e-4f8c-95a9-885dd350db4b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:16.157530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 343/685] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:16.157530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15456066-634e-4f8c-95a9-885dd350db4b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:16.172892 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc7731-7454-4a94-8d79-df8862b4cfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:16.173850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc7731-7454-4a94-8d79-df8862b4cfae None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:16.174323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc7731-7454-4a94-8d79-df8862b4cfae None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:16.174780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc7731-7454-4a94-8d79-df8862b4cfae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:16.185877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc7731-7454-4a94-8d79-df8862b4cfae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:16.187577 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc7731-7454-4a94-8d79-df8862b4cfae None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-8abc7731-7454-4a94-8d79-df8862b4cfae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:16.203543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:16.204643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:16.205856 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:16.205856 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:16.205856 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:16.205856 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:16.213352 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 343/686] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:16.214671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc7731-7454-4a94-8d79-df8862b4cfae None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:16.242370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:16.264315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15456066-634e-4f8c-95a9-885dd350db4b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:16.293358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:16.301435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3ace8-4388-4c0f-8d82-7cded15e0bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:16.302109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3ace8-4388-4c0f-8d82-7cded15e0bda None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:16.302396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3ace8-4388-4c0f-8d82-7cded15e0bda None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:16.302675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3ace8-4388-4c0f-8d82-7cded15e0bda None admin] PATH_INFO: `/v3/users/nova` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:16.310344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3ace8-4388-4c0f-8d82-7cded15e0bda None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:16.311786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3ace8-4388-4c0f-8d82-7cded15e0bda None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-7ea3ace8-4388-4c0f-8d82-7cded15e0bda None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:16.324540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:16.325778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:16.326860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:16.326860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:16.326860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:16.326860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:16.326860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:16.326860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:16.326860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:16.326860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:16.326860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:16.326860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:16.326860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 344/687] 10.222.0.73 () {56 vars in 1093 bytes} [Thu Aug 4 21:22:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:16.327519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3ace8-4388-4c0f-8d82-7cded15e0bda None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:16.333412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc7731-7454-4a94-8d79-df8862b4cfae None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:16.370099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c08fca-8236-4aab-92f5-a0edb175b0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:16.371002 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08fca-8236-4aab-92f5-a0edb175b0ad None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:16.371441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08fca-8236-4aab-92f5-a0edb175b0ad None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:16.371861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08fca-8236-4aab-92f5-a0edb175b0ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:16.372802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c08fca-8236-4aab-92f5-a0edb175b0ad None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=c98761f81496456fb3b5a5cffd4ce763) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:16.373410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c08fca-8236-4aab-92f5-a0edb175b0ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:16.376133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c08fca-8236-4aab-92f5-a0edb175b0ad None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:16.389523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 344/688] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:16 2022] GET /identity/v3/role_assignments?user.id=c98761f81496456fb3b5a5cffd4ce763&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:16.391892 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c08fca-8236-4aab-92f5-a0edb175b0ad None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:16.440684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3ace8-4388-4c0f-8d82-7cded15e0bda None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:16.480842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1499f6d-f315-4bef-b509-0f2d1d92a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:16.481533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1499f6d-f315-4bef-b509-0f2d1d92a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:16.481755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1499f6d-f315-4bef-b509-0f2d1d92a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:16.482043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1499f6d-f315-4bef-b509-0f2d1d92a3e2 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:16.482886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1499f6d-f315-4bef-b509-0f2d1d92a3e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:16.483329 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1499f6d-f315-4bef-b509-0f2d1d92a3e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:16.484677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1499f6d-f315-4bef-b509-0f2d1d92a3e2 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:16.499444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:16.501660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 345/689] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:16.509247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1499f6d-f315-4bef-b509-0f2d1d92a3e2 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:16.531075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c08fca-8236-4aab-92f5-a0edb175b0ad None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:16.571223 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c6fd0-2f60-462c-8462-12c6c5405319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:16.571973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c6fd0-2f60-462c-8462-12c6c5405319 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:16.572261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c6fd0-2f60-462c-8462-12c6c5405319 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:16.572535 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c6fd0-2f60-462c-8462-12c6c5405319 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:16.573438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c6fd0-2f60-462c-8462-12c6c5405319 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:16.573894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c6fd0-2f60-462c-8462-12c6c5405319 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:16.575481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c6fd0-2f60-462c-8462-12c6c5405319 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:16.588892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:16.596800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 345/690] 10.222.0.73 () {56 vars in 1133 bytes} [Thu Aug 4 21:22:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:16.607132 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c6fd0-2f60-462c-8462-12c6c5405319 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:16.668562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1499f6d-f315-4bef-b509-0f2d1d92a3e2 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:16.718909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf647e-7fee-4828-9f74-c036b5f4c5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:16.719825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf647e-7fee-4828-9f74-c036b5f4c5c0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:16.720103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf647e-7fee-4828-9f74-c036b5f4c5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:16.720374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf647e-7fee-4828-9f74-c036b5f4c5c0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:16.733689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf647e-7fee-4828-9f74-c036b5f4c5c0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:16.735171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf647e-7fee-4828-9f74-c036b5f4c5c0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-d7cf647e-7fee-4828-9f74-c036b5f4c5c0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:16.750923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:16.752052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:16.753846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:16.753846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:16.753846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:16.753846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:16.753846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:16.753846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:16.753846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:16.753846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:16.753846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:16.753846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:16.753846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 346/691] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:16.760969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf647e-7fee-4828-9f74-c036b5f4c5c0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:16.766633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c6fd0-2f60-462c-8462-12c6c5405319 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:16.807425 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f58e4-5379-4605-ae60-4ea2dd6e060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:16.808055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f58e4-5379-4605-ae60-4ea2dd6e060a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:16.808338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f58e4-5379-4605-ae60-4ea2dd6e060a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:16.808619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f58e4-5379-4605-ae60-4ea2dd6e060a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:16.813102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:16.897415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf647e-7fee-4828-9f74-c036b5f4c5c0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:16.908725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f58e4-5379-4605-ae60-4ea2dd6e060a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:16.909383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:16.911731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f58e4-5379-4605-ae60-4ea2dd6e060a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:16.935366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f9e91-adaf-408d-8e65-139455647619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:16.940650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f9e91-adaf-408d-8e65-139455647619 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:16.940650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f9e91-adaf-408d-8e65-139455647619 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:16.940650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f9e91-adaf-408d-8e65-139455647619 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:16.940650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f9e91-adaf-408d-8e65-139455647619 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:16.940650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f9e91-adaf-408d-8e65-139455647619 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:16.940650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f9e91-adaf-408d-8e65-139455647619 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:16.959892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 347/692] 10.222.0.73 () {56 vars in 1135 bytes} [Thu Aug 4 21:22:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:16.971559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f9e91-adaf-408d-8e65-139455647619 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:17.033333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 346/693] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:22:17.035299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ced13-e2da-401e-a0a1-aecdfc37212d None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:17.035645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ced13-e2da-401e-a0a1-aecdfc37212d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:17.035980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ced13-e2da-401e-a0a1-aecdfc37212d None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:17.037566 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 347/694] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:17.039486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ced13-e2da-401e-a0a1-aecdfc37212d None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:17.088492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:17.107471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f9e91-adaf-408d-8e65-139455647619 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:17.148934 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f9055-d2f9-428d-b094-fa4b86d85680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:17.149856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f9055-d2f9-428d-b094-fa4b86d85680 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:17.150366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f9055-d2f9-428d-b094-fa4b86d85680 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:17.151038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f9055-d2f9-428d-b094-fa4b86d85680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:17.167524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ced13-e2da-401e-a0a1-aecdfc37212d None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:17.190480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:17.214579 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78491ce1-c9ff-4259-ad37-028f93bd0907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:17.216049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78491ce1-c9ff-4259-ad37-028f93bd0907 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:17.216494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78491ce1-c9ff-4259-ad37-028f93bd0907 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:17.216930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78491ce1-c9ff-4259-ad37-028f93bd0907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:17.230943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f9055-d2f9-428d-b094-fa4b86d85680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:17.232027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78491ce1-c9ff-4259-ad37-028f93bd0907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:17.232709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f9055-d2f9-428d-b094-fa4b86d85680 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:17.233775 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78491ce1-c9ff-4259-ad37-028f93bd0907 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-78491ce1-c9ff-4259-ad37-028f93bd0907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:17.250167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:17.251163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:17.252117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:17.252117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:17.252117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:17.252117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:17.254402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 348/695] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:17.255790 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b327fcb-6f48-49c9-8db2-e927308c7a49 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:17.256298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b327fcb-6f48-49c9-8db2-e927308c7a49 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:17.256737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b327fcb-6f48-49c9-8db2-e927308c7a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:17.276363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-7b327fcb-6f48-49c9-8db2-e927308c7a49 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:17.277860 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-7b327fcb-6f48-49c9-8db2-e927308c7a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:17.354030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 348/696] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:22:17.354030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f9055-d2f9-428d-b094-fa4b86d85680 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:17.354624 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 349/697] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:17.355717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b327fcb-6f48-49c9-8db2-e927308c7a49 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:17.390336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:17.429049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:17.467376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f9055-d2f9-428d-b094-fa4b86d85680 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:17.469218 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b327fcb-6f48-49c9-8db2-e927308c7a49 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:17.505679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44988b62-8d9b-4b64-b884-cc419111e4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:17.506398 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54468c63-e30d-4e67-972c-7256211ff520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:17.506398 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54468c63-e30d-4e67-972c-7256211ff520 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:17.506822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54468c63-e30d-4e67-972c-7256211ff520 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:17.506907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54468c63-e30d-4e67-972c-7256211ff520 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:17.507741 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54468c63-e30d-4e67-972c-7256211ff520 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:17.507973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44988b62-8d9b-4b64-b884-cc419111e4ba None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:17.508120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54468c63-e30d-4e67-972c-7256211ff520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:17.508559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44988b62-8d9b-4b64-b884-cc419111e4ba None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:17.509015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44988b62-8d9b-4b64-b884-cc419111e4ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:17.509718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54468c63-e30d-4e67-972c-7256211ff520 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:17.528390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 349/698] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:17.529455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54468c63-e30d-4e67-972c-7256211ff520 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:17.532714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44988b62-8d9b-4b64-b884-cc419111e4ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:17.535325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44988b62-8d9b-4b64-b884-cc419111e4ba None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-44988b62-8d9b-4b64-b884-cc419111e4ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:17.555451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:17.556448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:17.557353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:17.557353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:17.557353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:17.557353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:17.563968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 350/699] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:17.563968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44988b62-8d9b-4b64-b884-cc419111e4ba None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:17.635454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:17.667190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44988b62-8d9b-4b64-b884-cc419111e4ba None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:17.671672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54468c63-e30d-4e67-972c-7256211ff520 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:17.708371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6864261-024a-498c-bdd5-bcee75bdd7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:17.709094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6864261-024a-498c-bdd5-bcee75bdd7d5 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:17.709365 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6864261-024a-498c-bdd5-bcee75bdd7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:17.709694 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6864261-024a-498c-bdd5-bcee75bdd7d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:17.710612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:17.712620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c9fee-0b5a-4184-8abf-d1a863be2803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:17.713799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c9fee-0b5a-4184-8abf-d1a863be2803 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:17.714286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c9fee-0b5a-4184-8abf-d1a863be2803 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:17.714739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c9fee-0b5a-4184-8abf-d1a863be2803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:17.724069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6864261-024a-498c-bdd5-bcee75bdd7d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:17.724267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c9fee-0b5a-4184-8abf-d1a863be2803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:17.724389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6864261-024a-498c-bdd5-bcee75bdd7d5 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:17.726038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c9fee-0b5a-4184-8abf-d1a863be2803 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-d6864261-024a-498c-bdd5-bcee75bdd7d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:17.740083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:17.741050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:17.742035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:17.742035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:17.742035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:17.742035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:17.742035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 350/700] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:17.742035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6864261-024a-498c-bdd5-bcee75bdd7d5 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-354c9fee-0b5a-4184-8abf-d1a863be2803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:17.743296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:17.744403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:17.745355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:17.745355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:17.745355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:17.745355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:17.745355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:17.745355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:17.745355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:17.745355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:17.748850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 351/701] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:17.749905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c9fee-0b5a-4184-8abf-d1a863be2803 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:17.857232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c9fee-0b5a-4184-8abf-d1a863be2803 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:17.878285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6864261-024a-498c-bdd5-bcee75bdd7d5 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:17.888542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:17.895757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff5821-df5b-455f-838b-36d76f6be8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:17.896763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff5821-df5b-455f-838b-36d76f6be8ef None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:17.897054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff5821-df5b-455f-838b-36d76f6be8ef None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:17.897442 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff5821-df5b-455f-838b-36d76f6be8ef None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:17.898133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff5821-df5b-455f-838b-36d76f6be8ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:17.901172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff5821-df5b-455f-838b-36d76f6be8ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:17.901172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff5821-df5b-455f-838b-36d76f6be8ef None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:17.914558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 352/702] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:17.914558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff5821-df5b-455f-838b-36d76f6be8ef None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:17.921529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7590f-9394-4e1b-9b9f-858f99dfb0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:17.922230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7590f-9394-4e1b-9b9f-858f99dfb0d3 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:17.922518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7590f-9394-4e1b-9b9f-858f99dfb0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:17.922793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7590f-9394-4e1b-9b9f-858f99dfb0d3 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:17.923496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7590f-9394-4e1b-9b9f-858f99dfb0d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:17.923803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7590f-9394-4e1b-9b9f-858f99dfb0d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:17.926206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7590f-9394-4e1b-9b9f-858f99dfb0d3 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:17.944999 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 351/703] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:17.946420 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ef68-667e-4506-9234-28d80e4284da None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:17.952010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ef68-667e-4506-9234-28d80e4284da None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:17.952010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ef68-667e-4506-9234-28d80e4284da None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:17.952010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 352/704] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:17.952010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6ef68-667e-4506-9234-28d80e4284da None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:17.995398 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:18.035513 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff5821-df5b-455f-838b-36d76f6be8ef None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:18.056956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6ef68-667e-4506-9234-28d80e4284da None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:18.069497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8d657-16de-49d0-97b2-fd255ac31032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:18.070149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8d657-16de-49d0-97b2-fd255ac31032 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:18.070426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8d657-16de-49d0-97b2-fd255ac31032 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:18.070696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8d657-16de-49d0-97b2-fd255ac31032 None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:18.071428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8d657-16de-49d0-97b2-fd255ac31032 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:18.071729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8d657-16de-49d0-97b2-fd255ac31032 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:18.073135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8d657-16de-49d0-97b2-fd255ac31032 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:18.089903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 353/705] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:18.090767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8d657-16de-49d0-97b2-fd255ac31032 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:18.092655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae5a0a-1dfa-454c-b25a-ee4a68800509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:18.093279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae5a0a-1dfa-454c-b25a-ee4a68800509 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:18.093565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae5a0a-1dfa-454c-b25a-ee4a68800509 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:18.093843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae5a0a-1dfa-454c-b25a-ee4a68800509 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:18.094545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae5a0a-1dfa-454c-b25a-ee4a68800509 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:18.095961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae5a0a-1dfa-454c-b25a-ee4a68800509 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:18.141073 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:18.180351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 353/706] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:22:17 2022] POST /identity/v3/users => generated 254 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:22:18.181592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71fb6-2569-4031-82ad-61ec862bf623 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:18.181879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71fb6-2569-4031-82ad-61ec862bf623 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:18.182315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71fb6-2569-4031-82ad-61ec862bf623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:18.202111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8d657-16de-49d0-97b2-fd255ac31032 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:18.208037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-b9c71fb6-2569-4031-82ad-61ec862bf623 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:18.209369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b9c71fb6-2569-4031-82ad-61ec862bf623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:18.246025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d415b19-9db7-45af-8c7a-cd9964eb56bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:18.247260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d415b19-9db7-45af-8c7a-cd9964eb56bd None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:18.247706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d415b19-9db7-45af-8c7a-cd9964eb56bd None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:18.248144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d415b19-9db7-45af-8c7a-cd9964eb56bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:18.258087 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d415b19-9db7-45af-8c7a-cd9964eb56bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:18.260582 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d415b19-9db7-45af-8c7a-cd9964eb56bd None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-9d415b19-9db7-45af-8c7a-cd9964eb56bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:18.278430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:18.279574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:18.280680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:18.280680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:18.280680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:18.280680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:18.280680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:18.280680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:18.280680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:18.280680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:18.283977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 354/707] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:18.285150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d415b19-9db7-45af-8c7a-cd9964eb56bd None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:18.304055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 354/708] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:18.305268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c71fb6-2569-4031-82ad-61ec862bf623 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:18.334614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:18.419354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d415b19-9db7-45af-8c7a-cd9964eb56bd None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:18.431616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:18.447266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c71fb6-2569-4031-82ad-61ec862bf623 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:18.469035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db767518-563a-49c1-88aa-712f3ac71bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:18.469769 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db767518-563a-49c1-88aa-712f3ac71bc8 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:18.472377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db767518-563a-49c1-88aa-712f3ac71bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:18.472700 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db767518-563a-49c1-88aa-712f3ac71bc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:18.473652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db767518-563a-49c1-88aa-712f3ac71bc8 None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=e7b8267780364a2181abf5817b54ce5c) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:18.473983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db767518-563a-49c1-88aa-712f3ac71bc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:18.475573 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db767518-563a-49c1-88aa-712f3ac71bc8 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:18.488578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 355/709] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:18 2022] GET /identity/v3/role_assignments?user.id=e7b8267780364a2181abf5817b54ce5c&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:18.489741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db767518-563a-49c1-88aa-712f3ac71bc8 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:18.509427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d5b77-fa36-40ca-b0cf-39101a177d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:18.510052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d5b77-fa36-40ca-b0cf-39101a177d0b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:18.510932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d5b77-fa36-40ca-b0cf-39101a177d0b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:18.510932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d5b77-fa36-40ca-b0cf-39101a177d0b None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:18.511439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d5b77-fa36-40ca-b0cf-39101a177d0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:18.511755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d5b77-fa36-40ca-b0cf-39101a177d0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:18.513352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d5b77-fa36-40ca-b0cf-39101a177d0b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:18.541349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 355/710] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:18.548726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d5b77-fa36-40ca-b0cf-39101a177d0b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:18.699216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db767518-563a-49c1-88aa-712f3ac71bc8 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:18.711026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:18.723007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d5b77-fa36-40ca-b0cf-39101a177d0b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:18.763160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29881611-e6aa-468d-800f-5edfec162edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:18.764082 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29881611-e6aa-468d-800f-5edfec162edd None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:18.764461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29881611-e6aa-468d-800f-5edfec162edd None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:18.764848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29881611-e6aa-468d-800f-5edfec162edd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:18.766188 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29881611-e6aa-468d-800f-5edfec162edd None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=3342e1039131496ba121b04685ec7b3d) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:18.766601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29881611-e6aa-468d-800f-5edfec162edd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:18.768991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29881611-e6aa-468d-800f-5edfec162edd None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:18.776611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 356/711] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:18 2022] GET /identity/v3/role_assignments?user.id=3342e1039131496ba121b04685ec7b3d&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:18.795588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc554ea2-7063-4565-b2a4-5aecc56c5820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:18.796514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc554ea2-7063-4565-b2a4-5aecc56c5820 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:18.797007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc554ea2-7063-4565-b2a4-5aecc56c5820 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:18.797455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc554ea2-7063-4565-b2a4-5aecc56c5820 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:18.798276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc554ea2-7063-4565-b2a4-5aecc56c5820 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:18.800327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc554ea2-7063-4565-b2a4-5aecc56c5820 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-bc554ea2-7063-4565-b2a4-5aecc56c5820 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:18.820714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:18.821939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:18.825594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 356/712] 10.222.0.73 () {56 vars in 1129 bytes} [Thu Aug 4 21:22:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:18.830384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29881611-e6aa-468d-800f-5edfec162edd None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:18.890054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487881d8-6fe3-42ad-bafe-c17cc4735312 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:18.890388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487881d8-6fe3-42ad-bafe-c17cc4735312 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:18.890780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487881d8-6fe3-42ad-bafe-c17cc4735312 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:18.892080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 357/713] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:22:18.898662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7851f-2b3f-4cde-a030-cbc9d13801e2 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:18.899336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7851f-2b3f-4cde-a030-cbc9d13801e2 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:18.899865 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7851f-2b3f-4cde-a030-cbc9d13801e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:18.920109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-baf7851f-2b3f-4cde-a030-cbc9d13801e2 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:18.921806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-baf7851f-2b3f-4cde-a030-cbc9d13801e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:18.965834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:18.978457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29881611-e6aa-468d-800f-5edfec162edd None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:19.021492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 358/714] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 21:22:19.043948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7851f-2b3f-4cde-a030-cbc9d13801e2 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:19.055302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac70ee5-be69-4ecd-adc2-913ce05aca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:19.055941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac70ee5-be69-4ecd-adc2-913ce05aca2d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:19.056372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac70ee5-be69-4ecd-adc2-913ce05aca2d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:19.056657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac70ee5-be69-4ecd-adc2-913ce05aca2d None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:19.057395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac70ee5-be69-4ecd-adc2-913ce05aca2d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:19.057705 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac70ee5-be69-4ecd-adc2-913ce05aca2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:19.059329 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac70ee5-be69-4ecd-adc2-913ce05aca2d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:19.064964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 357/715] 10.222.0.73 () {56 vars in 1124 bytes} [Thu Aug 4 21:22:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:19.071838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac70ee5-be69-4ecd-adc2-913ce05aca2d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:19.090425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:19.165489 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7851f-2b3f-4cde-a030-cbc9d13801e2 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:19.196533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac70ee5-be69-4ecd-adc2-913ce05aca2d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:19.207476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f28131-5e39-499a-8af9-39dbe91716fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:19.208133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f28131-5e39-499a-8af9-39dbe91716fc None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:19.208782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f28131-5e39-499a-8af9-39dbe91716fc None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:19.209057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f28131-5e39-499a-8af9-39dbe91716fc None admin] PATH_INFO: `/v3/services/network` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:19.209690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f28131-5e39-499a-8af9-39dbe91716fc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:19.211667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f28131-5e39-499a-8af9-39dbe91716fc None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-26f28131-5e39-499a-8af9-39dbe91716fc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:19.223802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:19.224850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:22:19.225680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:19.226476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 359/716] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:19 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:19.231993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f28131-5e39-499a-8af9-39dbe91716fc None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:19.246962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c184fc-c789-4f4d-a9fa-4c25fe51180a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:19.247604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c184fc-c789-4f4d-a9fa-4c25fe51180a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:19.248153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c184fc-c789-4f4d-a9fa-4c25fe51180a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:19.248431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c184fc-c789-4f4d-a9fa-4c25fe51180a None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:19.249162 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c184fc-c789-4f4d-a9fa-4c25fe51180a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:19.249624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c184fc-c789-4f4d-a9fa-4c25fe51180a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:19.251715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c184fc-c789-4f4d-a9fa-4c25fe51180a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:19.259926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 358/717] 10.222.0.73 () {56 vars in 1124 bytes} [Thu Aug 4 21:22:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:19.278724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:19.364511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f28131-5e39-499a-8af9-39dbe91716fc None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:19.406144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d0829-805b-4157-955f-4b4dff12d88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:19.406787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d0829-805b-4157-955f-4b4dff12d88a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:19.407069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d0829-805b-4157-955f-4b4dff12d88a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:19.407342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d0829-805b-4157-955f-4b4dff12d88a None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:19.408036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d0829-805b-4157-955f-4b4dff12d88a None admin] RBAC: Adding query filter params (name=network) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:19.408337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d0829-805b-4157-955f-4b4dff12d88a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:19.409939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d0829-805b-4157-955f-4b4dff12d88a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:19.415627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 360/718] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:19.421440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c184fc-c789-4f4d-a9fa-4c25fe51180a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:19.511576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:19.535682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c184fc-c789-4f4d-a9fa-4c25fe51180a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:19.571739 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15035cbc-8e56-469c-bfed-d7f4315d6a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:19.572441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15035cbc-8e56-469c-bfed-d7f4315d6a64 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:19.572720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15035cbc-8e56-469c-bfed-d7f4315d6a64 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:19.573043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15035cbc-8e56-469c-bfed-d7f4315d6a64 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:19.573726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15035cbc-8e56-469c-bfed-d7f4315d6a64 None admin] RBAC: Adding query filter params (type=network) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:19.574048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15035cbc-8e56-469c-bfed-d7f4315d6a64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:19.575522 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15035cbc-8e56-469c-bfed-d7f4315d6a64 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:19.580504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 359/719] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:20.614050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c008a9-c94f-4ae3-a1f4-87d45a2b81d9 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:20.614585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c008a9-c94f-4ae3-a1f4-87d45a2b81d9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:20.614585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c008a9-c94f-4ae3-a1f4-87d45a2b81d9 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:20.616193 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 361/720] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:20.621575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965da5c-d289-4cfb-a7c0-b93b834f129d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:20.622053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965da5c-d289-4cfb-a7c0-b93b834f129d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:20.622134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965da5c-d289-4cfb-a7c0-b93b834f129d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:20.648033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-5965da5c-d289-4cfb-a7c0-b93b834f129d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:20.649229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5965da5c-d289-4cfb-a7c0-b93b834f129d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:20.732757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 360/721] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:20.769501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c008a9-c94f-4ae3-a1f4-87d45a2b81d9 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:20.883151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:20.894657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c008a9-c94f-4ae3-a1f4-87d45a2b81d9 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:20.926199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857283c-d5d7-4419-8e7c-a92939baf711 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:20.926512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857283c-d5d7-4419-8e7c-a92939baf711 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:20.926753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857283c-d5d7-4419-8e7c-a92939baf711 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:20.928253 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 361/722] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:22:20.935530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b0d47-3a4e-4ad5-9180-c91bfe62a664 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:20.935812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b0d47-3a4e-4ad5-9180-c91bfe62a664 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:20.935917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723d6e2-ea1f-49cd-acce-b12a5a6d12c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:20.936143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b0d47-3a4e-4ad5-9180-c91bfe62a664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:20.936629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723d6e2-ea1f-49cd-acce-b12a5a6d12c2 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:20.936914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723d6e2-ea1f-49cd-acce-b12a5a6d12c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:20.937188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723d6e2-ea1f-49cd-acce-b12a5a6d12c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:20.961058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723d6e2-ea1f-49cd-acce-b12a5a6d12c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-8723d6e2-ea1f-49cd-acce-b12a5a6d12c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:20.966245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:20.967197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:20.968054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 362/723] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:20.975021 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1e3b0d47-3a4e-4ad5-9180-c91bfe62a664 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:20.975257 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723d6e2-ea1f-49cd-acce-b12a5a6d12c2 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:20.976240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1e3b0d47-3a4e-4ad5-9180-c91bfe62a664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:21.101188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723d6e2-ea1f-49cd-acce-b12a5a6d12c2 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:21.123325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:21.138142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 362/724] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:21.139943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e755acd-af11-4009-9306-ccefe822328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:21.140802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e755acd-af11-4009-9306-ccefe822328d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:21.141294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e755acd-af11-4009-9306-ccefe822328d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:21.141767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e755acd-af11-4009-9306-ccefe822328d None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:21.142737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e755acd-af11-4009-9306-ccefe822328d None admin] RBAC: Adding query filter params (name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:21.143782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e755acd-af11-4009-9306-ccefe822328d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:21.145517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e755acd-af11-4009-9306-ccefe822328d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:21.154760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 363/725] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:21.162821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b0d47-3a4e-4ad5-9180-c91bfe62a664 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:21.164021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e755acd-af11-4009-9306-ccefe822328d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:21.201006 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:21.289579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b0d47-3a4e-4ad5-9180-c91bfe62a664 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:21.326632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52983dc-7533-47ba-bb93-767ac6f40b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:21.327697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52983dc-7533-47ba-bb93-767ac6f40b4f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:21.328181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52983dc-7533-47ba-bb93-767ac6f40b4f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:21.328797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52983dc-7533-47ba-bb93-767ac6f40b4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:21.340412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52983dc-7533-47ba-bb93-767ac6f40b4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:21.342158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52983dc-7533-47ba-bb93-767ac6f40b4f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-d52983dc-7533-47ba-bb93-767ac6f40b4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:21.357900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:21.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:21.359833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:21.359833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:21.359833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:21.359833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:21.362556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 363/726] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:21.368205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52983dc-7533-47ba-bb93-767ac6f40b4f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:21.424844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:21.441394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e755acd-af11-4009-9306-ccefe822328d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:21.477239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52983dc-7533-47ba-bb93-767ac6f40b4f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:21.513878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd874b9-74f2-4e94-ab9a-574e17485d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:21.514883 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd874b9-74f2-4e94-ab9a-574e17485d77 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:21.515377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd874b9-74f2-4e94-ab9a-574e17485d77 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:21.515850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd874b9-74f2-4e94-ab9a-574e17485d77 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:21.516792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd874b9-74f2-4e94-ab9a-574e17485d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:21.528479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd874b9-74f2-4e94-ab9a-574e17485d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:21.530245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd874b9-74f2-4e94-ab9a-574e17485d77 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:21.544182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 364/727] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:21.550077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd874b9-74f2-4e94-ab9a-574e17485d77 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:21.570927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5f787-2c68-47d2-ac17-1b1775ff5df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:21.571405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5f787-2c68-47d2-ac17-1b1775ff5df7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:21.571689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5f787-2c68-47d2-ac17-1b1775ff5df7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:21.571964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5f787-2c68-47d2-ac17-1b1775ff5df7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:21.618681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5f787-2c68-47d2-ac17-1b1775ff5df7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-a9c5f787-2c68-47d2-ac17-1b1775ff5df7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:21.623989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:21.625021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:21.625906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 364/728] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:21.640072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5f787-2c68-47d2-ac17-1b1775ff5df7 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:21.664365 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd874b9-74f2-4e94-ab9a-574e17485d77 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:21.679767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:21.702541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71065fcf-05fd-4696-8cc5-4b3c3e8a3226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:21.703549 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065fcf-05fd-4696-8cc5-4b3c3e8a3226 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:21.708051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065fcf-05fd-4696-8cc5-4b3c3e8a3226 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:21.708051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065fcf-05fd-4696-8cc5-4b3c3e8a3226 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:21.712726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71065fcf-05fd-4696-8cc5-4b3c3e8a3226 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:21.714541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71065fcf-05fd-4696-8cc5-4b3c3e8a3226 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-71065fcf-05fd-4696-8cc5-4b3c3e8a3226 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:21.728933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:21.729945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:21.730910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:21.730910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:21.730910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:21.730910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:21.730910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:21.730910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:21.730910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:21.730910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:21.730910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:21.730910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:21.733576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 365/729] 10.222.0.73 () {56 vars in 1117 bytes} [Thu Aug 4 21:22:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:21.739550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71065fcf-05fd-4696-8cc5-4b3c3e8a3226 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:21.814405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:21.836456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5f787-2c68-47d2-ac17-1b1775ff5df7 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:21.848550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71065fcf-05fd-4696-8cc5-4b3c3e8a3226 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:21.872785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a29ea7-3d8d-4f3d-8674-5f8aee433ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:21.873477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a29ea7-3d8d-4f3d-8674-5f8aee433ec7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:21.873831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a29ea7-3d8d-4f3d-8674-5f8aee433ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:21.874193 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a29ea7-3d8d-4f3d-8674-5f8aee433ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:21.874976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a29ea7-3d8d-4f3d-8674-5f8aee433ec7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:21.875289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a29ea7-3d8d-4f3d-8674-5f8aee433ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:21.876648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a29ea7-3d8d-4f3d-8674-5f8aee433ec7 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:21.885807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 365/730] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:22:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:21.886139 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab15b7-2e0b-4441-9a30-141ea2775c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:21.886948 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab15b7-2e0b-4441-9a30-141ea2775c55 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:21.887055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ce94b-6647-4624-9472-54f6ae6804bc None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:21.887308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ce94b-6647-4624-9472-54f6ae6804bc None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:21.887589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ce94b-6647-4624-9472-54f6ae6804bc None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:21.887898 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab15b7-2e0b-4441-9a30-141ea2775c55 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:21.888352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab15b7-2e0b-4441-9a30-141ea2775c55 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:21.888761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 366/731] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:21.889530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab15b7-2e0b-4441-9a30-141ea2775c55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:21.889758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f028da-2d0c-4a15-9998-87eab8497665 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:21.890055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f028da-2d0c-4a15-9998-87eab8497665 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:21.890374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab15b7-2e0b-4441-9a30-141ea2775c55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:21.890569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f028da-2d0c-4a15-9998-87eab8497665 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:21.891979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 367/732] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:21.892300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab15b7-2e0b-4441-9a30-141ea2775c55 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:21.892673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f028da-2d0c-4a15-9998-87eab8497665 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:21.924921 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 366/733] 10.222.0.73 () {56 vars in 1149 bytes} [Thu Aug 4 21:22:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:21.926623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedce48-fa84-4c31-9feb-8ad2c19c7e75 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:21.926916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedce48-fa84-4c31-9feb-8ad2c19c7e75 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:21.927190 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedce48-fa84-4c31-9feb-8ad2c19c7e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:21.945841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1cedce48-fa84-4c31-9feb-8ad2c19c7e75 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:21.947095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1cedce48-fa84-4c31-9feb-8ad2c19c7e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:21.971870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:22.001176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f028da-2d0c-4a15-9998-87eab8497665 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:22.026033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 367/734] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:22.026033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3cd97-8d2f-4fe8-aff7-e845d38f30f6 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:22.026033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3cd97-8d2f-4fe8-aff7-e845d38f30f6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:22.026033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3cd97-8d2f-4fe8-aff7-e845d38f30f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:22.043816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bce3cd97-8d2f-4fe8-aff7-e845d38f30f6 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:22.045056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bce3cd97-8d2f-4fe8-aff7-e845d38f30f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:22.045912 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece6783-2076-4a8c-b11c-60470a256910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:22.046957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece6783-2076-4a8c-b11c-60470a256910 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:22.047243 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece6783-2076-4a8c-b11c-60470a256910 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:22.047517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece6783-2076-4a8c-b11c-60470a256910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:22.056668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:22.059150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece6783-2076-4a8c-b11c-60470a256910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:22.060594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece6783-2076-4a8c-b11c-60470a256910 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-dece6783-2076-4a8c-b11c-60470a256910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:22.075338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:22.076271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:22.077177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:22.077177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:22.077177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:22.077177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:22.077177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 368/735] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:22.077542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece6783-2076-4a8c-b11c-60470a256910 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:22.122096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 368/736] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:22.123254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3cd97-8d2f-4fe8-aff7-e845d38f30f6 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:22.196000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece6783-2076-4a8c-b11c-60470a256910 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:22.230873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:22.231779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a38895-2711-471d-9135-a1a712bb39bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:22.232469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a38895-2711-471d-9135-a1a712bb39bd None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:22.232754 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a38895-2711-471d-9135-a1a712bb39bd None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:22.233025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a38895-2711-471d-9135-a1a712bb39bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:22.241104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3cd97-8d2f-4fe8-aff7-e845d38f30f6 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:22.276751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c36bc-ee33-43a9-b3f2-eeb64f41d7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:22.277387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c36bc-ee33-43a9-b3f2-eeb64f41d7fb None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:22.277671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c36bc-ee33-43a9-b3f2-eeb64f41d7fb None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:22.277950 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c36bc-ee33-43a9-b3f2-eeb64f41d7fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:22.288800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c36bc-ee33-43a9-b3f2-eeb64f41d7fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:22.293004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-6e2c36bc-ee33-43a9-b3f2-eeb64f41d7fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:22.293584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:22.294544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:22.295396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 369/737] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:22.296089 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c36bc-ee33-43a9-b3f2-eeb64f41d7fb None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:22.306238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a38895-2711-471d-9135-a1a712bb39bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:22.308289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a38895-2711-471d-9135-a1a712bb39bd None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:22.403506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c36bc-ee33-43a9-b3f2-eeb64f41d7fb None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:22.405454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 369/738] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 21:22:22.406402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a38895-2711-471d-9135-a1a712bb39bd None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:22.437530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44fa3b-fd15-45e9-bd3c-2de27c9f0e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:22.438172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44fa3b-fd15-45e9-bd3c-2de27c9f0e37 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:22.438454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44fa3b-fd15-45e9-bd3c-2de27c9f0e37 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:22.438727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44fa3b-fd15-45e9-bd3c-2de27c9f0e37 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:22.439430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44fa3b-fd15-45e9-bd3c-2de27c9f0e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:22.439732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44fa3b-fd15-45e9-bd3c-2de27c9f0e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:22.441839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44fa3b-fd15-45e9-bd3c-2de27c9f0e37 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:22.452953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 370/739] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:22.454061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc0859-035d-4b40-b6a2-86f952afafa0 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:22.454343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc0859-035d-4b40-b6a2-86f952afafa0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:22.454616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc0859-035d-4b40-b6a2-86f952afafa0 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:22.455926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 371/740] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:22.456520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc0859-035d-4b40-b6a2-86f952afafa0 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:22.469121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:22.501414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:22.510581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a38895-2711-471d-9135-a1a712bb39bd None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:22.544922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c1a91-9933-4a2f-bc04-26b520ad18c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:22.545648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c1a91-9933-4a2f-bc04-26b520ad18c7 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:22.545931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c1a91-9933-4a2f-bc04-26b520ad18c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:22.546232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c1a91-9933-4a2f-bc04-26b520ad18c7 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:22.546928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c1a91-9933-4a2f-bc04-26b520ad18c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:22.548292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c1a91-9933-4a2f-bc04-26b520ad18c7 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:22.560865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc0859-035d-4b40-b6a2-86f952afafa0 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:22.594357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523c34f-1dc5-4141-9658-e9f4c1a5a9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:22.595012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523c34f-1dc5-4141-9658-e9f4c1a5a9af None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:22.595326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523c34f-1dc5-4141-9658-e9f4c1a5a9af None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:22.595600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523c34f-1dc5-4141-9658-e9f4c1a5a9af None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:22.596351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523c34f-1dc5-4141-9658-e9f4c1a5a9af None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:22.596643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523c34f-1dc5-4141-9658-e9f4c1a5a9af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:22.597966 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523c34f-1dc5-4141-9658-e9f4c1a5a9af None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:22.605803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 372/741] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:22:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:22.606598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523c34f-1dc5-4141-9658-e9f4c1a5a9af None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:22.655604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 370/742] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:22:22 2022] POST /identity/v3/services => generated 251 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:22:22.656544 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c1a91-9933-4a2f-bc04-26b520ad18c7 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:22.679853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:22.724066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523c34f-1dc5-4141-9658-e9f4c1a5a9af None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:22.780307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b81df3-3bbc-4605-9af3-0a0cc55c69b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:22.780969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81df3-3bbc-4605-9af3-0a0cc55c69b9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:22.781252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81df3-3bbc-4605-9af3-0a0cc55c69b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:22.781526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81df3-3bbc-4605-9af3-0a0cc55c69b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:22.783556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c1a91-9933-4a2f-bc04-26b520ad18c7 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:22.796571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b81df3-3bbc-4605-9af3-0a0cc55c69b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:22.798084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b81df3-3bbc-4605-9af3-0a0cc55c69b9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-40b81df3-3bbc-4605-9af3-0a0cc55c69b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:22.815346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:22.816313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:22.817655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:22.817655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:22.817655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:22.817655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:22.817655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 373/743] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:22.817943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c50cf-3fa9-4a68-9485-ea63fb238258 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:22.818035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c50cf-3fa9-4a68-9485-ea63fb238258 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:22.818534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c50cf-3fa9-4a68-9485-ea63fb238258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:22.825627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632136f-5081-4e74-808d-fda1b2e5497d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:22.826306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632136f-5081-4e74-808d-fda1b2e5497d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:22.826588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632136f-5081-4e74-808d-fda1b2e5497d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:22.826861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632136f-5081-4e74-808d-fda1b2e5497d None admin] PATH_INFO: `/v3/users/nova` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:22.835297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632136f-5081-4e74-808d-fda1b2e5497d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:22.836741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632136f-5081-4e74-808d-fda1b2e5497d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:22.838141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ac3c50cf-3fa9-4a68-9485-ea63fb238258 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:22.839351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ac3c50cf-3fa9-4a68-9485-ea63fb238258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-e632136f-5081-4e74-808d-fda1b2e5497d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:22.851586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:22.852616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:22.853550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:22.853550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:22.853550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:22.853550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:22.853550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:22.853550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:22.853550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:22.853550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:22.853550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:22.853550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:22.853550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 371/744] 10.222.0.73 () {56 vars in 1093 bytes} [Thu Aug 4 21:22:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:22.854171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632136f-5081-4e74-808d-fda1b2e5497d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:22.866792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:22.923035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 374/745] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 21:22:22.924470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c50cf-3fa9-4a68-9485-ea63fb238258 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:22.972030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632136f-5081-4e74-808d-fda1b2e5497d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:22.984552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:23.010758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af625d-04f1-43d5-92e1-50704e1079c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:23.011727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af625d-04f1-43d5-92e1-50704e1079c9 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:23.012010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af625d-04f1-43d5-92e1-50704e1079c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:23.012287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af625d-04f1-43d5-92e1-50704e1079c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:23.024551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af625d-04f1-43d5-92e1-50704e1079c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:23.026040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af625d-04f1-43d5-92e1-50704e1079c9 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-96af625d-04f1-43d5-92e1-50704e1079c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:23.041510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:23.042547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:23.043517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:23.043517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:23.043517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:23.043517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:23.045786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c50cf-3fa9-4a68-9485-ea63fb238258 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:23.047463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 372/746] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:23.048083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af625d-04f1-43d5-92e1-50704e1079c9 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:23.085124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995c472-5f9f-4e45-b16f-b55b3d58bafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:23.085752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995c472-5f9f-4e45-b16f-b55b3d58bafd None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:23.086059 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995c472-5f9f-4e45-b16f-b55b3d58bafd None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:23.086345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995c472-5f9f-4e45-b16f-b55b3d58bafd None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:23.087070 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995c472-5f9f-4e45-b16f-b55b3d58bafd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:23.087380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995c472-5f9f-4e45-b16f-b55b3d58bafd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:23.088773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995c472-5f9f-4e45-b16f-b55b3d58bafd None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:23.101887 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 375/747] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:23.102603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995c472-5f9f-4e45-b16f-b55b3d58bafd None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:23.118128 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:23.163777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af625d-04f1-43d5-92e1-50704e1079c9 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:23.199736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55121af-d8da-4390-b4fc-4b4df4c5ea82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:23.200360 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55121af-d8da-4390-b4fc-4b4df4c5ea82 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:23.200675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55121af-d8da-4390-b4fc-4b4df4c5ea82 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:23.200955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55121af-d8da-4390-b4fc-4b4df4c5ea82 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:23.201751 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55121af-d8da-4390-b4fc-4b4df4c5ea82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:23.202100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55121af-d8da-4390-b4fc-4b4df4c5ea82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:23.203528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55121af-d8da-4390-b4fc-4b4df4c5ea82 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:23.209639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995c472-5f9f-4e45-b16f-b55b3d58bafd None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:23.220271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:23.223062 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 373/748] 10.222.0.73 () {56 vars in 1133 bytes} [Thu Aug 4 21:22:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:23.223992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55121af-d8da-4390-b4fc-4b4df4c5ea82 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:23.249905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba95fd-22cd-44cf-9fab-ade6e13931a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:23.250493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba95fd-22cd-44cf-9fab-ade6e13931a6 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:23.250857 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba95fd-22cd-44cf-9fab-ade6e13931a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:23.251046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba95fd-22cd-44cf-9fab-ade6e13931a6 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:23.251734 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba95fd-22cd-44cf-9fab-ade6e13931a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:23.253728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba95fd-22cd-44cf-9fab-ade6e13931a6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:23.324065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 376/749] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:22:23 2022] POST /identity/v3/services => generated 240 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:22:23.325150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55121af-d8da-4390-b4fc-4b4df4c5ea82 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:23.325386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba95fd-22cd-44cf-9fab-ade6e13931a6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:23.355010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:23.386492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a74ca-0656-40a1-a857-0b327827d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:23.387561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a74ca-0656-40a1-a857-0b327827d67d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:23.387848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a74ca-0656-40a1-a857-0b327827d67d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:23.388120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a74ca-0656-40a1-a857-0b327827d67d None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:23.388850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a74ca-0656-40a1-a857-0b327827d67d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:23.389153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a74ca-0656-40a1-a857-0b327827d67d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:23.391640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a74ca-0656-40a1-a857-0b327827d67d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:23.415559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 374/750] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:23.416926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a74ca-0656-40a1-a857-0b327827d67d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:23.463727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba95fd-22cd-44cf-9fab-ade6e13931a6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:23.507085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974f461-4a0c-4402-bf42-7f56247f3bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:23.507943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974f461-4a0c-4402-bf42-7f56247f3bbf None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:23.508228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974f461-4a0c-4402-bf42-7f56247f3bbf None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:23.508510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974f461-4a0c-4402-bf42-7f56247f3bbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:23.518931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974f461-4a0c-4402-bf42-7f56247f3bbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:23.520385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974f461-4a0c-4402-bf42-7f56247f3bbf None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-a974f461-4a0c-4402-bf42-7f56247f3bbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:23.532630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:23.533588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:23.535222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:23.535222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:23.535222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:23.535222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:23.535222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:23.535222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:23.535222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:23.535222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:23.535222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 377/751] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:23.535741 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974f461-4a0c-4402-bf42-7f56247f3bbf None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:23.574195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a74ca-0656-40a1-a857-0b327827d67d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:23.576835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:23.619914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee167fbd-d3dd-4243-ba63-7e17ed78d25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:23.620969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee167fbd-d3dd-4243-ba63-7e17ed78d25b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:23.621460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee167fbd-d3dd-4243-ba63-7e17ed78d25b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:23.621944 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee167fbd-d3dd-4243-ba63-7e17ed78d25b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:23.635328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee167fbd-d3dd-4243-ba63-7e17ed78d25b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:23.637107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee167fbd-d3dd-4243-ba63-7e17ed78d25b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-ee167fbd-d3dd-4243-ba63-7e17ed78d25b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:23.656563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:23.657930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:23.658887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:23.658887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:23.658887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:23.658887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:23.661332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 375/752] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:23.662390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee167fbd-d3dd-4243-ba63-7e17ed78d25b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:23.680675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974f461-4a0c-4402-bf42-7f56247f3bbf None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:23.696972 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:23.725696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fff1b4-7c8f-4f9c-b5f7-7d21365681ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:23.726376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fff1b4-7c8f-4f9c-b5f7-7d21365681ef None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:23.726660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fff1b4-7c8f-4f9c-b5f7-7d21365681ef None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:23.726956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fff1b4-7c8f-4f9c-b5f7-7d21365681ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:23.736926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fff1b4-7c8f-4f9c-b5f7-7d21365681ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:23.738652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fff1b4-7c8f-4f9c-b5f7-7d21365681ef None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-50fff1b4-7c8f-4f9c-b5f7-7d21365681ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:23.754334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:23.755333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:23.756273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:23.756273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:23.756273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:23.756273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:23.756273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:23.756273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:23.756273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:23.756273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:23.756273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:23.756273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:23.763470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 378/753] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:23.764723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fff1b4-7c8f-4f9c-b5f7-7d21365681ef None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:23.787346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee167fbd-d3dd-4243-ba63-7e17ed78d25b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:23.834327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a305b98-d8f8-4740-b1d6-e131cff0bc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:23.834957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a305b98-d8f8-4740-b1d6-e131cff0bc4a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:23.835238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a305b98-d8f8-4740-b1d6-e131cff0bc4a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:23.835518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a305b98-d8f8-4740-b1d6-e131cff0bc4a None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:23.836860 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a305b98-d8f8-4740-b1d6-e131cff0bc4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:23.837175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a305b98-d8f8-4740-b1d6-e131cff0bc4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:23.838652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a305b98-d8f8-4740-b1d6-e131cff0bc4a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:23.854411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 376/754] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:23.855600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a305b98-d8f8-4740-b1d6-e131cff0bc4a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:23.870246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:23.879955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fff1b4-7c8f-4f9c-b5f7-7d21365681ef None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:23.915236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182267f-25dd-4456-89e1-f04f83aa5b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:23.915858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182267f-25dd-4456-89e1-f04f83aa5b37 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:23.916160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182267f-25dd-4456-89e1-f04f83aa5b37 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:23.916443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182267f-25dd-4456-89e1-f04f83aa5b37 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:23.917431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182267f-25dd-4456-89e1-f04f83aa5b37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:23.917745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182267f-25dd-4456-89e1-f04f83aa5b37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:23.919414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182267f-25dd-4456-89e1-f04f83aa5b37 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:23.933827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 379/755] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:23.934857 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182267f-25dd-4456-89e1-f04f83aa5b37 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:23.978495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:24.000645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a305b98-d8f8-4740-b1d6-e131cff0bc4a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:24.041378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4205d86-fb50-4ef0-9b52-a740449a5c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:24.041905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4205d86-fb50-4ef0-9b52-a740449a5c54 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:24.042287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4205d86-fb50-4ef0-9b52-a740449a5c54 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:24.042559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4205d86-fb50-4ef0-9b52-a740449a5c54 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:24.043344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4205d86-fb50-4ef0-9b52-a740449a5c54 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:24.043650 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4205d86-fb50-4ef0-9b52-a740449a5c54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:24.045073 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4205d86-fb50-4ef0-9b52-a740449a5c54 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:24.053349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182267f-25dd-4456-89e1-f04f83aa5b37 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:24.068392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 377/756] 10.222.0.73 () {56 vars in 1135 bytes} [Thu Aug 4 21:22:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:24.068392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4205d86-fb50-4ef0-9b52-a740449a5c54 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:24.105731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19407866-073e-450e-a772-f4263d9f85eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:24.106664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19407866-073e-450e-a772-f4263d9f85eb None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:24.107098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19407866-073e-450e-a772-f4263d9f85eb None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:24.107522 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19407866-073e-450e-a772-f4263d9f85eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:24.109043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19407866-073e-450e-a772-f4263d9f85eb None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=6027cfb8625c4d028e7160a85e81bb74) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:24.109521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19407866-073e-450e-a772-f4263d9f85eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:24.112443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19407866-073e-450e-a772-f4263d9f85eb None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:24.121535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 380/757] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:23 2022] GET /identity/v3/role_assignments?user.id=6027cfb8625c4d028e7160a85e81bb74&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:24.122567 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19407866-073e-450e-a772-f4263d9f85eb None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:24.141336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:24.243571 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4205d86-fb50-4ef0-9b52-a740449a5c54 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:24.253558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19407866-073e-450e-a772-f4263d9f85eb None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:24.297081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd3f15-302f-4e9a-91eb-92da839861f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:24.297960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd3f15-302f-4e9a-91eb-92da839861f7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:24.298418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd3f15-302f-4e9a-91eb-92da839861f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:24.298843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd3f15-302f-4e9a-91eb-92da839861f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:24.333448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57172b54-8abb-4d28-ad8b-1af3e41ae334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:24.334213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57172b54-8abb-4d28-ad8b-1af3e41ae334 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:24.334493 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57172b54-8abb-4d28-ad8b-1af3e41ae334 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:24.334762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57172b54-8abb-4d28-ad8b-1af3e41ae334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:24.342261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57172b54-8abb-4d28-ad8b-1af3e41ae334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:24.343739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57172b54-8abb-4d28-ad8b-1af3e41ae334 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-57172b54-8abb-4d28-ad8b-1af3e41ae334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:24.375661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:24.376635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:24.378177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:24.378177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:24.378177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:24.378177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:24.378177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:24.378177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:24.378177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:24.378177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:24.378177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 378/758] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:24.382911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57172b54-8abb-4d28-ad8b-1af3e41ae334 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:24.391987 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:24.394589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd3f15-302f-4e9a-91eb-92da839861f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:24.396263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd3f15-302f-4e9a-91eb-92da839861f7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:24.414145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:24.528673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 381/759] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:22:24.534917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd3f15-302f-4e9a-91eb-92da839861f7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:24.656098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd3f15-302f-4e9a-91eb-92da839861f7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:24.668446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57172b54-8abb-4d28-ad8b-1af3e41ae334 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:24.672792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:24.697722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e755ee4-f271-41df-aeb2-5b8d594333d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:24.699187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e755ee4-f271-41df-aeb2-5b8d594333d5 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:24.699675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e755ee4-f271-41df-aeb2-5b8d594333d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:24.700149 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e755ee4-f271-41df-aeb2-5b8d594333d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:24.712437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e755ee4-f271-41df-aeb2-5b8d594333d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:24.714222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e755ee4-f271-41df-aeb2-5b8d594333d5 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-0e755ee4-f271-41df-aeb2-5b8d594333d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:24.730217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:24.731150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:24.732044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:24.732044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:24.732044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:24.732044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:24.734357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 382/760] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:24.741179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e755ee4-f271-41df-aeb2-5b8d594333d5 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:24.771667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766754e-db1e-4c32-97d5-b5d3476c1d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:24.772299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766754e-db1e-4c32-97d5-b5d3476c1d30 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:24.772584 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766754e-db1e-4c32-97d5-b5d3476c1d30 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:24.772858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766754e-db1e-4c32-97d5-b5d3476c1d30 None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:24.773605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766754e-db1e-4c32-97d5-b5d3476c1d30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:24.773919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766754e-db1e-4c32-97d5-b5d3476c1d30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:24.775383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766754e-db1e-4c32-97d5-b5d3476c1d30 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:24.804210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 379/761] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:24.810115 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766754e-db1e-4c32-97d5-b5d3476c1d30 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:24.874864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e755ee4-f271-41df-aeb2-5b8d594333d5 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:24.881976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:24.920093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c99fb-5b2f-4f42-a289-8378a45ea41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:24.921223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c99fb-5b2f-4f42-a289-8378a45ea41e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:24.922240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c99fb-5b2f-4f42-a289-8378a45ea41e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:24.922737 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c99fb-5b2f-4f42-a289-8378a45ea41e None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:24.923922 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c99fb-5b2f-4f42-a289-8378a45ea41e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:24.924910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c99fb-5b2f-4f42-a289-8378a45ea41e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:24.927185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c99fb-5b2f-4f42-a289-8378a45ea41e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:24.931833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766754e-db1e-4c32-97d5-b5d3476c1d30 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:24.947221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 383/762] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:24.947790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f384db1-de0a-4377-a7a0-82a416069a8d None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:24.947900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f384db1-de0a-4377-a7a0-82a416069a8d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:24.948098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f384db1-de0a-4377-a7a0-82a416069a8d None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:24.949504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 384/763] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:24.952049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f384db1-de0a-4377-a7a0-82a416069a8d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:24.985255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:24.991066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e3638-20ee-4388-ba83-309ac50d7c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:24.992209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e3638-20ee-4388-ba83-309ac50d7c9d None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:24.992742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e3638-20ee-4388-ba83-309ac50d7c9d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:24.993275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e3638-20ee-4388-ba83-309ac50d7c9d None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/e7b8267780364a2181abf5817b54ce5c/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:24.996457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e3638-20ee-4388-ba83-309ac50d7c9d None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=e7b8267780364a2181abf5817b54ce5c, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:25.000619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e3638-20ee-4388-ba83-309ac50d7c9d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:25.065460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f384db1-de0a-4377-a7a0-82a416069a8d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:25.074480 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 380/764] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:22:24 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/e7b8267780364a2181abf5817b54ce5c/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:22:25.076574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020e72-1339-4f1b-9914-ec5b46f87fda None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:25.077509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020e72-1339-4f1b-9914-ec5b46f87fda None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:25.078305 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020e72-1339-4f1b-9914-ec5b46f87fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:25.103851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-b6020e72-1339-4f1b-9914-ec5b46f87fda None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:25.105702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b6020e72-1339-4f1b-9914-ec5b46f87fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:25.121002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cde762-9d06-433f-8bfa-565ec1106d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:25.121709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cde762-9d06-433f-8bfa-565ec1106d72 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:25.122148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cde762-9d06-433f-8bfa-565ec1106d72 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:25.122564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cde762-9d06-433f-8bfa-565ec1106d72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:25.135832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cde762-9d06-433f-8bfa-565ec1106d72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:25.137340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cde762-9d06-433f-8bfa-565ec1106d72 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-85cde762-9d06-433f-8bfa-565ec1106d72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:25.161704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:25.162722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:25.163615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:25.163615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:25.163615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:25.163615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:25.163615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:25.163615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:25.163615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:25.163615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:25.163615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 385/765] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:25.172116 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cde762-9d06-433f-8bfa-565ec1106d72 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:25.203067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 381/766] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:25.230230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6020e72-1339-4f1b-9914-ec5b46f87fda None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:25.303550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:25.351500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:25.360622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6020e72-1339-4f1b-9914-ec5b46f87fda None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:25.379047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cde762-9d06-433f-8bfa-565ec1106d72 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:25.411768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c782e-611f-4e15-b347-dddff03b0b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:25.412501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c782e-611f-4e15-b347-dddff03b0b33 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:25.412824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c782e-611f-4e15-b347-dddff03b0b33 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:25.413102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c782e-611f-4e15-b347-dddff03b0b33 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:25.413907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c782e-611f-4e15-b347-dddff03b0b33 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:25.415439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c782e-611f-4e15-b347-dddff03b0b33 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-1c8c782e-611f-4e15-b347-dddff03b0b33 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:25.427403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:25.428391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:25.429275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 382/767] 10.222.0.73 () {56 vars in 1129 bytes} [Thu Aug 4 21:22:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:25.437906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb6c21-213b-45b8-b173-ab33fdda05f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:25.438157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c782e-611f-4e15-b347-dddff03b0b33 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:25.438674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6c21-213b-45b8-b173-ab33fdda05f4 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:25.438965 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6c21-213b-45b8-b173-ab33fdda05f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:25.439238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6c21-213b-45b8-b173-ab33fdda05f4 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:25.440142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb6c21-213b-45b8-b173-ab33fdda05f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:25.440456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb6c21-213b-45b8-b173-ab33fdda05f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:25.441945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb6c21-213b-45b8-b173-ab33fdda05f4 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:25.456950 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 386/768] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:25.463405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb6c21-213b-45b8-b173-ab33fdda05f4 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:25.575612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb6c21-213b-45b8-b173-ab33fdda05f4 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:25.621671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e757f6-b174-4579-b8cc-6fe5b1f5b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:25.622340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e757f6-b174-4579-b8cc-6fe5b1f5b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:25.622625 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e757f6-b174-4579-b8cc-6fe5b1f5b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:25.622901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e757f6-b174-4579-b8cc-6fe5b1f5b8c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:25.623703 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e757f6-b174-4579-b8cc-6fe5b1f5b8c6 None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=3342e1039131496ba121b04685ec7b3d) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:25.624016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e757f6-b174-4579-b8cc-6fe5b1f5b8c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:25.625452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e757f6-b174-4579-b8cc-6fe5b1f5b8c6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:25.641591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 387/769] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:25 2022] GET /identity/v3/role_assignments?user.id=3342e1039131496ba121b04685ec7b3d&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:25.643789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c782e-611f-4e15-b347-dddff03b0b33 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:25.669118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:25.680929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669298f-7426-45be-898c-a3fca930512d None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:25.681257 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669298f-7426-45be-898c-a3fca930512d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:25.681487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669298f-7426-45be-898c-a3fca930512d None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:25.682886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 388/770] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:22:25.686771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba481704-b84c-41cc-a916-f1d16c5ed9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:25.687399 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba481704-b84c-41cc-a916-f1d16c5ed9ff None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:25.687779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba481704-b84c-41cc-a916-f1d16c5ed9ff None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:25.688046 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba481704-b84c-41cc-a916-f1d16c5ed9ff None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:25.688737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba481704-b84c-41cc-a916-f1d16c5ed9ff None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:25.689043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba481704-b84c-41cc-a916-f1d16c5ed9ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:25.691033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba481704-b84c-41cc-a916-f1d16c5ed9ff None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:25.698870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43491435-3c24-4d62-bed1-3526fb8a8143 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:25.698870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43491435-3c24-4d62-bed1-3526fb8a8143 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:25.698870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43491435-3c24-4d62-bed1-3526fb8a8143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:25.703579 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 383/771] 10.222.0.73 () {56 vars in 1124 bytes} [Thu Aug 4 21:22:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:25.710944 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba481704-b84c-41cc-a916-f1d16c5ed9ff None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:25.731565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-43491435-3c24-4d62-bed1-3526fb8a8143 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:25.733098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-43491435-3c24-4d62-bed1-3526fb8a8143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:25.740784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:25.834845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba481704-b84c-41cc-a916-f1d16c5ed9ff None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:25.835887 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 389/772] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:25.859898 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43491435-3c24-4d62-bed1-3526fb8a8143 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:25.878931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc01195-9e67-472e-8952-a063d9f6ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:25.879623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc01195-9e67-472e-8952-a063d9f6ac58 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:25.879914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc01195-9e67-472e-8952-a063d9f6ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:25.880130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc01195-9e67-472e-8952-a063d9f6ac58 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:25.880833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc01195-9e67-472e-8952-a063d9f6ac58 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:25.881139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc01195-9e67-472e-8952-a063d9f6ac58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:25.882672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc01195-9e67-472e-8952-a063d9f6ac58 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:25.898147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 384/773] 10.222.0.73 () {56 vars in 1124 bytes} [Thu Aug 4 21:22:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:25.905151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc01195-9e67-472e-8952-a063d9f6ac58 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:26.018617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43491435-3c24-4d62-bed1-3526fb8a8143 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:26.020285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:26.067245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10c4f8-d433-4ce8-aed8-e911a69bd1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:26.067910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10c4f8-d433-4ce8-aed8-e911a69bd1ab None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:26.068193 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10c4f8-d433-4ce8-aed8-e911a69bd1ab None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:26.068468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10c4f8-d433-4ce8-aed8-e911a69bd1ab None admin] PATH_INFO: `/v3/services/network` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:26.069109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10c4f8-d433-4ce8-aed8-e911a69bd1ab None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:26.070537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10c4f8-d433-4ce8-aed8-e911a69bd1ab None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-ab10c4f8-d433-4ce8-aed8-e911a69bd1ab None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:26.100660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:26.101744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:26.102641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 390/774] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:25 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:26.109614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10c4f8-d433-4ce8-aed8-e911a69bd1ab None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:26.118356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:26.142057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc01195-9e67-472e-8952-a063d9f6ac58 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:26.218030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c4b00-bcce-4450-9aa5-1f835ae627ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:26.218668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c4b00-bcce-4450-9aa5-1f835ae627ea None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:26.218954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c4b00-bcce-4450-9aa5-1f835ae627ea None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:26.219233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c4b00-bcce-4450-9aa5-1f835ae627ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:26.219999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c4b00-bcce-4450-9aa5-1f835ae627ea None admin] RBAC: Adding query filter params (interface=public, service_id=c6b9963ccba4411a859834ede3eb8db2, region_id=RegionOne) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:26.220311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c4b00-bcce-4450-9aa5-1f835ae627ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:26.221880 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c4b00-bcce-4450-9aa5-1f835ae627ea None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:26.246382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 385/775] 10.222.0.73 () {56 vars in 1251 bytes} [Thu Aug 4 21:22:25 2022] GET /identity/v3/endpoints?service_id=c6b9963ccba4411a859834ede3eb8db2&interface=public®ion_id=RegionOne => generated 194 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:26.247838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c4b00-bcce-4450-9aa5-1f835ae627ea None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:26.370490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c4b00-bcce-4450-9aa5-1f835ae627ea None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:26.384018 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10c4f8-d433-4ce8-aed8-e911a69bd1ab None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:26.390719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:26.427580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20677a8-9cb0-42da-b375-743599340245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:26.428578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20677a8-9cb0-42da-b375-743599340245 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:26.429057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20677a8-9cb0-42da-b375-743599340245 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:26.429642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20677a8-9cb0-42da-b375-743599340245 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:26.430648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20677a8-9cb0-42da-b375-743599340245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:26.432347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20677a8-9cb0-42da-b375-743599340245 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:26.435688 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a89fcd-19cd-4c38-bd12-84264d0fe58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:26.436230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a89fcd-19cd-4c38-bd12-84264d0fe58b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:26.436511 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a89fcd-19cd-4c38-bd12-84264d0fe58b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:26.436785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a89fcd-19cd-4c38-bd12-84264d0fe58b None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:26.437456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a89fcd-19cd-4c38-bd12-84264d0fe58b None admin] RBAC: Adding query filter params (name=network) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:26.437757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a89fcd-19cd-4c38-bd12-84264d0fe58b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:26.439131 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a89fcd-19cd-4c38-bd12-84264d0fe58b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:26.440819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 386/776] 10.222.0.73 () {56 vars in 1087 bytes} [Thu Aug 4 21:22:26 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:26.442484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521926a-4a78-4690-93b9-d7d3b1c522fc None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:26.442977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521926a-4a78-4690-93b9-d7d3b1c522fc None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:26.443448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521926a-4a78-4690-93b9-d7d3b1c522fc None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:26.449802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 391/777] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:26.450158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 387/778] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:26.451454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a89fcd-19cd-4c38-bd12-84264d0fe58b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:26.462288 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850fa5d-2015-4e50-ab77-1004ca0c426c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:26.462623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850fa5d-2015-4e50-ab77-1004ca0c426c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:26.462893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850fa5d-2015-4e50-ab77-1004ca0c426c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:26.503743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-3850fa5d-2015-4e50-ab77-1004ca0c426c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:26.506842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3850fa5d-2015-4e50-ab77-1004ca0c426c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:26.589006 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 388/779] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:26.603778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:26.616967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850fa5d-2015-4e50-ab77-1004ca0c426c None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:26.648688 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a89fcd-19cd-4c38-bd12-84264d0fe58b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:26.693418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d1a44-0d27-4abf-8396-4f3f9b436ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:26.694820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d1a44-0d27-4abf-8396-4f3f9b436ad0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:26.695209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d1a44-0d27-4abf-8396-4f3f9b436ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:26.695579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d1a44-0d27-4abf-8396-4f3f9b436ad0 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:26.696528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d1a44-0d27-4abf-8396-4f3f9b436ad0 None admin] RBAC: Adding query filter params (type=network) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:26.697067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d1a44-0d27-4abf-8396-4f3f9b436ad0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:26.698745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d1a44-0d27-4abf-8396-4f3f9b436ad0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:26.710069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 392/780] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:26.713019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d1a44-0d27-4abf-8396-4f3f9b436ad0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:26.745070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850fa5d-2015-4e50-ab77-1004ca0c426c None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:26.750344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:26.813885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35846cd-7b0a-4f71-8a89-3657c30c20cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:26.815348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35846cd-7b0a-4f71-8a89-3657c30c20cc None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:26.815951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35846cd-7b0a-4f71-8a89-3657c30c20cc None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:26.818730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35846cd-7b0a-4f71-8a89-3657c30c20cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:26.841384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35846cd-7b0a-4f71-8a89-3657c30c20cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-f35846cd-7b0a-4f71-8a89-3657c30c20cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:26.848527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:26.850036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:26.851074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:26.859063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 389/781] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:26.863172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35846cd-7b0a-4f71-8a89-3657c30c20cc None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:26.894087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d1a44-0d27-4abf-8396-4f3f9b436ad0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:26.951876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:26.957367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf06fb-3df1-44a5-8a95-b00ea2afee8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:26.957994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf06fb-3df1-44a5-8a95-b00ea2afee8d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:26.959462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf06fb-3df1-44a5-8a95-b00ea2afee8d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:26.959462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf06fb-3df1-44a5-8a95-b00ea2afee8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:26.960250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf06fb-3df1-44a5-8a95-b00ea2afee8d None admin] RBAC: Adding query filter params (interface=public, service_id=ba8945182bf049bcaa94c26666674ac9, region_id=RegionOne) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:26.960564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf06fb-3df1-44a5-8a95-b00ea2afee8d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:26.961920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf06fb-3df1-44a5-8a95-b00ea2afee8d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:26.983542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 393/782] 10.222.0.73 () {56 vars in 1251 bytes} [Thu Aug 4 21:22:26 2022] GET /identity/v3/endpoints?service_id=ba8945182bf049bcaa94c26666674ac9&interface=public®ion_id=RegionOne => generated 194 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:26.985524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf06fb-3df1-44a5-8a95-b00ea2afee8d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:27.070907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35846cd-7b0a-4f71-8a89-3657c30c20cc None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:27.132498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0aaee-26b8-426a-9b3a-fc6a12165608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:27.133473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0aaee-26b8-426a-9b3a-fc6a12165608 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:27.133994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0aaee-26b8-426a-9b3a-fc6a12165608 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:27.135469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0aaee-26b8-426a-9b3a-fc6a12165608 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:27.136425 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0aaee-26b8-426a-9b3a-fc6a12165608 None admin] RBAC: Adding query filter params (name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:27.138949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0aaee-26b8-426a-9b3a-fc6a12165608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:27.140571 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0aaee-26b8-426a-9b3a-fc6a12165608 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:27.164156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 390/783] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:27.166936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0aaee-26b8-426a-9b3a-fc6a12165608 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:27.204234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf06fb-3df1-44a5-8a95-b00ea2afee8d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:27.237721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:27.248310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9c226-52b2-4cf0-8c79-bc172b84cfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:27.249804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9c226-52b2-4cf0-8c79-bc172b84cfc9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:27.251445 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9c226-52b2-4cf0-8c79-bc172b84cfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:27.251909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9c226-52b2-4cf0-8c79-bc172b84cfc9 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:27.253433 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9c226-52b2-4cf0-8c79-bc172b84cfc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:27.257645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9c226-52b2-4cf0-8c79-bc172b84cfc9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:27.268328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 394/784] 10.222.0.73 () {56 vars in 1087 bytes} [Thu Aug 4 21:22:26 2022] GET /identity/v3/services => generated 772 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:27.353554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0aaee-26b8-426a-9b3a-fc6a12165608 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:27.397467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426997c-f57c-41a6-a258-bc27259dcc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:27.398476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426997c-f57c-41a6-a258-bc27259dcc1b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:27.398939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426997c-f57c-41a6-a258-bc27259dcc1b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:27.399388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426997c-f57c-41a6-a258-bc27259dcc1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:27.413254 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426997c-f57c-41a6-a258-bc27259dcc1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:27.415083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426997c-f57c-41a6-a258-bc27259dcc1b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-4426997c-f57c-41a6-a258-bc27259dcc1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:27.443122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:27.444108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:27.445011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:27.445011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:27.445011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:27.445011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:27.447141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 391/785] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:27.452752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9c226-52b2-4cf0-8c79-bc172b84cfc9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:27.520970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:27.564121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9c226-52b2-4cf0-8c79-bc172b84cfc9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:27.603726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9836d-2400-4139-bb4a-a50754f3bd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:27.604352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9836d-2400-4139-bb4a-a50754f3bd54 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:27.604633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9836d-2400-4139-bb4a-a50754f3bd54 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:27.604908 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9836d-2400-4139-bb4a-a50754f3bd54 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:27.605601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9836d-2400-4139-bb4a-a50754f3bd54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:27.605904 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9836d-2400-4139-bb4a-a50754f3bd54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:27.607282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9836d-2400-4139-bb4a-a50754f3bd54 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:27.623876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 395/786] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:27.629758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426997c-f57c-41a6-a258-bc27259dcc1b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:27.658539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26c931-11a3-4f37-ba21-ab981b80bf7a None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:27.659007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26c931-11a3-4f37-ba21-ab981b80bf7a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:27.659112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26c931-11a3-4f37-ba21-ab981b80bf7a None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:27.660690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 396/787] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:27.666735 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600fe03-558c-4ee9-9a57-a321e75e250e None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:27.667029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600fe03-558c-4ee9-9a57-a321e75e250e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:27.667274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600fe03-558c-4ee9-9a57-a321e75e250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:27.697967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-0600fe03-558c-4ee9-9a57-a321e75e250e None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:27.699173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-0600fe03-558c-4ee9-9a57-a321e75e250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:27.780569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:27.817515 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426997c-f57c-41a6-a258-bc27259dcc1b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:27.831666 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 397/788] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:27.875633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a42089-ee8c-4628-97d4-8b0d8ba0ac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:27.876285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a42089-ee8c-4628-97d4-8b0d8ba0ac75 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:27.876568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a42089-ee8c-4628-97d4-8b0d8ba0ac75 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:27.876839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a42089-ee8c-4628-97d4-8b0d8ba0ac75 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:27.882650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600fe03-558c-4ee9-9a57-a321e75e250e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:27.888042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a42089-ee8c-4628-97d4-8b0d8ba0ac75 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:27.889764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a42089-ee8c-4628-97d4-8b0d8ba0ac75 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-34a42089-ee8c-4628-97d4-8b0d8ba0ac75 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:27.913086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:27.916784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:27.917723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:27.917723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:27.917723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:27.917723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:27.917723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:27.917723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:27.917723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:27.917723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:27.917723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:27.917723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:27.921314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 392/789] 10.222.0.73 () {56 vars in 1117 bytes} [Thu Aug 4 21:22:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:27.927945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a42089-ee8c-4628-97d4-8b0d8ba0ac75 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:27.978373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:28.050393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600fe03-558c-4ee9-9a57-a321e75e250e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:28.084931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a42089-ee8c-4628-97d4-8b0d8ba0ac75 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:28.086409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f45fa-1a3e-4ebe-8963-0672b0c0e52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:28.087060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f45fa-1a3e-4ebe-8963-0672b0c0e52b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:28.087336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f45fa-1a3e-4ebe-8963-0672b0c0e52b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:28.087612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f45fa-1a3e-4ebe-8963-0672b0c0e52b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:28.099765 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f45fa-1a3e-4ebe-8963-0672b0c0e52b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-ea5f45fa-1a3e-4ebe-8963-0672b0c0e52b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:28.104853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:28.105840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:28.106713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:28.111502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 398/790] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:28.112967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a8a64-0486-48a5-837c-0943e4daab53 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:28.113256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a8a64-0486-48a5-837c-0943e4daab53 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:28.113535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a8a64-0486-48a5-837c-0943e4daab53 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:28.115188 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 399/791] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:28.115968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a8a64-0486-48a5-837c-0943e4daab53 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:28.131600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fdaa3-6712-4b55-be95-7577b38c1d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:28.132229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fdaa3-6712-4b55-be95-7577b38c1d1d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:28.132679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fdaa3-6712-4b55-be95-7577b38c1d1d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:28.132960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fdaa3-6712-4b55-be95-7577b38c1d1d None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:28.134196 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fdaa3-6712-4b55-be95-7577b38c1d1d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:28.134780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fdaa3-6712-4b55-be95-7577b38c1d1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:28.136702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fdaa3-6712-4b55-be95-7577b38c1d1d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:28.148485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:28.160999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 393/792] 10.222.0.73 () {56 vars in 1149 bytes} [Thu Aug 4 21:22:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:28.162490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c665673-b2b5-4432-8826-aa23614610f3 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:28.163275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c665673-b2b5-4432-8826-aa23614610f3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:28.166607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c665673-b2b5-4432-8826-aa23614610f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:28.188398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-8c665673-b2b5-4432-8826-aa23614610f3 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:28.190913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8c665673-b2b5-4432-8826-aa23614610f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:28.228669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a8a64-0486-48a5-837c-0943e4daab53 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:28.250397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:28.265878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b0ee9-93af-4194-bc2f-32a97876515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:28.266721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b0ee9-93af-4194-bc2f-32a97876515d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:28.267096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b0ee9-93af-4194-bc2f-32a97876515d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:28.267472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b0ee9-93af-4194-bc2f-32a97876515d None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:28.268435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b0ee9-93af-4194-bc2f-32a97876515d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:28.268718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b0ee9-93af-4194-bc2f-32a97876515d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:28.270251 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b0ee9-93af-4194-bc2f-32a97876515d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:28.286233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 400/794] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:22:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:28.286233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b0ee9-93af-4194-bc2f-32a97876515d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:28.286551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 394/794] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:28.287774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c665673-b2b5-4432-8826-aa23614610f3 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:28.415302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b0ee9-93af-4194-bc2f-32a97876515d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:28.420591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c665673-b2b5-4432-8826-aa23614610f3 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:28.454317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:28.459036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cf927-bf73-401a-9bf4-fd9623d94c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:28.460219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cf927-bf73-401a-9bf4-fd9623d94c61 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:28.460602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cf927-bf73-401a-9bf4-fd9623d94c61 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:28.460998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cf927-bf73-401a-9bf4-fd9623d94c61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:28.464744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baef1a8-e3a9-43b2-a156-fca6c5e6b04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:28.465609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baef1a8-e3a9-43b2-a156-fca6c5e6b04c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:28.466101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baef1a8-e3a9-43b2-a156-fca6c5e6b04c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:28.469703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baef1a8-e3a9-43b2-a156-fca6c5e6b04c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:28.472767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cf927-bf73-401a-9bf4-fd9623d94c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:28.474359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cf927-bf73-401a-9bf4-fd9623d94c61 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:28.479514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baef1a8-e3a9-43b2-a156-fca6c5e6b04c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:28.481186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baef1a8-e3a9-43b2-a156-fca6c5e6b04c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-471cf927-bf73-401a-9bf4-fd9623d94c61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:28.495268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:28.496236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:28.497140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:28.497140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:28.497140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:28.497140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:28.501540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 401/795] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:28.503139 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cf927-bf73-401a-9bf4-fd9623d94c61 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-3baef1a8-e3a9-43b2-a156-fca6c5e6b04c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:28.503484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:28.504354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:28.505249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:28.505249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:28.505249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:28.505249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:28.508510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 395/796] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:28.509627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baef1a8-e3a9-43b2-a156-fca6c5e6b04c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:28.535053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:28.616734 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cf927-bf73-401a-9bf4-fd9623d94c61 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:28.629443 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baef1a8-e3a9-43b2-a156-fca6c5e6b04c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:28.654491 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e29f47-9893-48f2-ad2c-da6087adc41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:28.655542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e29f47-9893-48f2-ad2c-da6087adc41c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:28.656104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e29f47-9893-48f2-ad2c-da6087adc41c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:28.656676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e29f47-9893-48f2-ad2c-da6087adc41c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:28.669329 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfb46f-a3fe-4654-a28c-99510a6cbbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:28.669860 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb46f-a3fe-4654-a28c-99510a6cbbc4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:28.670221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb46f-a3fe-4654-a28c-99510a6cbbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:28.670457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb46f-a3fe-4654-a28c-99510a6cbbc4 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:28.671279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e29f47-9893-48f2-ad2c-da6087adc41c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:28.671390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfb46f-a3fe-4654-a28c-99510a6cbbc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:28.671602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfb46f-a3fe-4654-a28c-99510a6cbbc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:28.673172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfb46f-a3fe-4654-a28c-99510a6cbbc4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-e4e29f47-9893-48f2-ad2c-da6087adc41c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:28.677454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:28.678423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:28.679284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:28.681823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 402/797] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:28.682863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e29f47-9893-48f2-ad2c-da6087adc41c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:28.696907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 396/798] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:28.697989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfb46f-a3fe-4654-a28c-99510a6cbbc4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:28.743007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:28.775991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:28.793261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e29f47-9893-48f2-ad2c-da6087adc41c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:28.810420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfb46f-a3fe-4654-a28c-99510a6cbbc4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:28.830393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41183b-4313-44a9-bba9-a287ebf63c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:28.831656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41183b-4313-44a9-bba9-a287ebf63c74 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:28.832205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41183b-4313-44a9-bba9-a287ebf63c74 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:28.833132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41183b-4313-44a9-bba9-a287ebf63c74 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:28.834046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41183b-4313-44a9-bba9-a287ebf63c74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:28.834574 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41183b-4313-44a9-bba9-a287ebf63c74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:28.836143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41183b-4313-44a9-bba9-a287ebf63c74 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:28.853017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44d494-1ebc-474c-ac10-d5bf208bb847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:28.853017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d494-1ebc-474c-ac10-d5bf208bb847 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:28.853017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d494-1ebc-474c-ac10-d5bf208bb847 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:28.853017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d494-1ebc-474c-ac10-d5bf208bb847 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:28.855075 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 403/799] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:28.856160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41183b-4313-44a9-bba9-a287ebf63c74 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:28.858728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44d494-1ebc-474c-ac10-d5bf208bb847 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:28.861883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44d494-1ebc-474c-ac10-d5bf208bb847 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-0e44d494-1ebc-474c-ac10-d5bf208bb847 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:28.878928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:28.879899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:28.880811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:28.880811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:28.880811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:28.880811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:28.880811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:28.880811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:28.880811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:28.880811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:28.885305 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 397/800] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:28.886388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44d494-1ebc-474c-ac10-d5bf208bb847 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:28.976221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41183b-4313-44a9-bba9-a287ebf63c74 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:29.010365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:29.013920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44d494-1ebc-474c-ac10-d5bf208bb847 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:29.015789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649d06c-4e5e-47b1-9c21-e2f961c48255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:29.016793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649d06c-4e5e-47b1-9c21-e2f961c48255 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.016793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649d06c-4e5e-47b1-9c21-e2f961c48255 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.021930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649d06c-4e5e-47b1-9c21-e2f961c48255 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.021930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649d06c-4e5e-47b1-9c21-e2f961c48255 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:29.021930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649d06c-4e5e-47b1-9c21-e2f961c48255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:29.021930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649d06c-4e5e-47b1-9c21-e2f961c48255 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:29.030879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 404/801] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:22:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:29.031724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649d06c-4e5e-47b1-9c21-e2f961c48255 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:29.041524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:29.058111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d9c13-b17b-429a-a6f7-8cc16066aa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:29.059116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d9c13-b17b-429a-a6f7-8cc16066aa54 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.059545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d9c13-b17b-429a-a6f7-8cc16066aa54 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.059963 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d9c13-b17b-429a-a6f7-8cc16066aa54 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.068898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d9c13-b17b-429a-a6f7-8cc16066aa54 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:29.070753 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d9c13-b17b-429a-a6f7-8cc16066aa54 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-a97d9c13-b17b-429a-a6f7-8cc16066aa54 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:29.083728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:29.084883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:29.086025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:29.086025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:29.086025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:29.086025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:29.086025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:29.086025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:29.086025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:29.086025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:29.086025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:29.086025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:29.088156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 398/802] 10.222.0.73 () {56 vars in 1093 bytes} [Thu Aug 4 21:22:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:29.089476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e1f8-6957-423c-9c91-639782124439 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.089759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e1f8-6957-423c-9c91-639782124439 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.090075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e1f8-6957-423c-9c91-639782124439 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.091494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 399/803] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:29.095093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13e1f8-6957-423c-9c91-639782124439 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:29.145958 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649d06c-4e5e-47b1-9c21-e2f961c48255 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:29.182584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e12a0-5f9f-4271-b12c-2181d27fb4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:29.183218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e12a0-5f9f-4271-b12c-2181d27fb4a6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.183497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e12a0-5f9f-4271-b12c-2181d27fb4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.183776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e12a0-5f9f-4271-b12c-2181d27fb4a6 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.184528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e12a0-5f9f-4271-b12c-2181d27fb4a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:29.184839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e12a0-5f9f-4271-b12c-2181d27fb4a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:29.186286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e12a0-5f9f-4271-b12c-2181d27fb4a6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:29.199957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 405/804] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:29.203025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e12a0-5f9f-4271-b12c-2181d27fb4a6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:29.204872 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13e1f8-6957-423c-9c91-639782124439 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:29.242652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7276c-c069-47f8-868c-3658e50cc1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:29.243202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7276c-c069-47f8-868c-3658e50cc1ff None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.243487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7276c-c069-47f8-868c-3658e50cc1ff None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.243761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7276c-c069-47f8-868c-3658e50cc1ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.255609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7276c-c069-47f8-868c-3658e50cc1ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:29.257048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7276c-c069-47f8-868c-3658e50cc1ff None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-f3a7276c-c069-47f8-868c-3658e50cc1ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:29.271965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:29.273015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:29.273923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:29.273923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:29.273923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:29.273923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:29.273923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 400/805] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:29.274687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741d30a-e1c6-4f09-a9cd-17e917e33c27 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.274687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741d30a-e1c6-4f09-a9cd-17e917e33c27 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.274813 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741d30a-e1c6-4f09-a9cd-17e917e33c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.281356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:29.293834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-9741d30a-e1c6-4f09-a9cd-17e917e33c27 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:29.295078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9741d30a-e1c6-4f09-a9cd-17e917e33c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:29.298643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:29.319931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e12a0-5f9f-4271-b12c-2181d27fb4a6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:29.354947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc2aae-8bfc-4ac3-9a31-f7f6f1ab3fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:29.355644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc2aae-8bfc-4ac3-9a31-f7f6f1ab3fb3 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.355932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc2aae-8bfc-4ac3-9a31-f7f6f1ab3fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.356225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc2aae-8bfc-4ac3-9a31-f7f6f1ab3fb3 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.357014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc2aae-8bfc-4ac3-9a31-f7f6f1ab3fb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:29.357321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc2aae-8bfc-4ac3-9a31-f7f6f1ab3fb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:29.358826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc2aae-8bfc-4ac3-9a31-f7f6f1ab3fb3 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:29.370800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 401/806] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:29.374829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741d30a-e1c6-4f09-a9cd-17e917e33c27 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:29.383461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 406/807] 10.222.0.73 () {56 vars in 1133 bytes} [Thu Aug 4 21:22:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:29.384384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc2aae-8bfc-4ac3-9a31-f7f6f1ab3fb3 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:29.490215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741d30a-e1c6-4f09-a9cd-17e917e33c27 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:29.494602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc2aae-8bfc-4ac3-9a31-f7f6f1ab3fb3 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:29.524380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:29.536978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb5d2a-e2af-4682-8e63-1c6841f936d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:29.537601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb5d2a-e2af-4682-8e63-1c6841f936d9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.540194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb5d2a-e2af-4682-8e63-1c6841f936d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.540303 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0cdac-d131-4b5b-b91f-f9dc9190507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:29.540424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb5d2a-e2af-4682-8e63-1c6841f936d9 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.540516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0cdac-d131-4b5b-b91f-f9dc9190507d None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.540516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0cdac-d131-4b5b-b91f-f9dc9190507d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.540683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb5d2a-e2af-4682-8e63-1c6841f936d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:29.540773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0cdac-d131-4b5b-b91f-f9dc9190507d None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/6027cfb8625c4d028e7160a85e81bb74/roles/280f24425d0a4b16951d30c20d7290f9` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.540858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb5d2a-e2af-4682-8e63-1c6841f936d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:29.540858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb5d2a-e2af-4682-8e63-1c6841f936d9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:29.552815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 407/808] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:29.554623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:29.554759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb5d2a-e2af-4682-8e63-1c6841f936d9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:29.557349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0cdac-d131-4b5b-b91f-f9dc9190507d None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=6027cfb8625c4d028e7160a85e81bb74, role_id=280f24425d0a4b16951d30c20d7290f9)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:29.558824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0cdac-d131-4b5b-b91f-f9dc9190507d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:29.630095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 402/809] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:22:29 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/6027cfb8625c4d028e7160a85e81bb74/roles/280f24425d0a4b16951d30c20d7290f9 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:22:29.630960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0cdac-d131-4b5b-b91f-f9dc9190507d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:29.668808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb5d2a-e2af-4682-8e63-1c6841f936d9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:29.708574 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24197812-d646-4d2b-a205-8a5a7b90c884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:29.709481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24197812-d646-4d2b-a205-8a5a7b90c884 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.709770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24197812-d646-4d2b-a205-8a5a7b90c884 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.710091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24197812-d646-4d2b-a205-8a5a7b90c884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.759968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0cdac-d131-4b5b-b91f-f9dc9190507d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:29.775931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:29.797832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24197812-d646-4d2b-a205-8a5a7b90c884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:29.800248 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24197812-d646-4d2b-a205-8a5a7b90c884 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:29.811859 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548e1a9-092d-4c68-a630-9b42a6ee18a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:29.812598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548e1a9-092d-4c68-a630-9b42a6ee18a6 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.812784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548e1a9-092d-4c68-a630-9b42a6ee18a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.813152 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548e1a9-092d-4c68-a630-9b42a6ee18a6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.813793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548e1a9-092d-4c68-a630-9b42a6ee18a6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:29.815225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548e1a9-092d-4c68-a630-9b42a6ee18a6 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-b548e1a9-092d-4c68-a630-9b42a6ee18a6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:29.825653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:29.826886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:29.827921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 403/810] 10.222.0.73 () {56 vars in 1129 bytes} [Thu Aug 4 21:22:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:29.828978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78640e6e-ce9b-41f0-b76e-6970ac1f7215 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.828978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78640e6e-ce9b-41f0-b76e-6970ac1f7215 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.828978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78640e6e-ce9b-41f0-b76e-6970ac1f7215 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.830030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 404/811] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:29.837159 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78640e6e-ce9b-41f0-b76e-6970ac1f7215 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:29.906821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:29.921864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 408/812] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:22:29.924062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b6ab5-2d31-4896-87f3-cdc4ff07923d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:29.924544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b6ab5-2d31-4896-87f3-cdc4ff07923d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:29.925029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b6ab5-2d31-4896-87f3-cdc4ff07923d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:29.947330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-377b6ab5-2d31-4896-87f3-cdc4ff07923d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:29.948529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-377b6ab5-2d31-4896-87f3-cdc4ff07923d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:29.966848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78640e6e-ce9b-41f0-b76e-6970ac1f7215 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:30.009882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ec747-a643-42aa-bf19-100012c1261a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:30.011085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ec747-a643-42aa-bf19-100012c1261a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:30.011390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ec747-a643-42aa-bf19-100012c1261a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:30.011724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ec747-a643-42aa-bf19-100012c1261a None admin] PATH_INFO: `/v3/users/swift` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:30.025304 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ec747-a643-42aa-bf19-100012c1261a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:30.026815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ec747-a643-42aa-bf19-100012c1261a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:30.037195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 409/813] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:30.038310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b6ab5-2d31-4896-87f3-cdc4ff07923d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-778ec747-a643-42aa-bf19-100012c1261a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:30.040685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:30.046366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 405/814] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:30.047459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ec747-a643-42aa-bf19-100012c1261a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:30.071185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:30.164729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b6ab5-2d31-4896-87f3-cdc4ff07923d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:30.183480 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ec747-a643-42aa-bf19-100012c1261a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:30.200875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:30.208420 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ff2ae-cd36-4c62-9c5d-3e64bcc14e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:30.209274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ff2ae-cd36-4c62-9c5d-3e64bcc14e22 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:30.215593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ff2ae-cd36-4c62-9c5d-3e64bcc14e22 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:30.215593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ff2ae-cd36-4c62-9c5d-3e64bcc14e22 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:30.215593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ff2ae-cd36-4c62-9c5d-3e64bcc14e22 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:30.215593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ff2ae-cd36-4c62-9c5d-3e64bcc14e22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:30.215593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ff2ae-cd36-4c62-9c5d-3e64bcc14e22 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:30.226568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 410/815] 10.222.0.73 () {56 vars in 1124 bytes} [Thu Aug 4 21:22:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:30.226568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ff2ae-cd36-4c62-9c5d-3e64bcc14e22 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:30.230988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145deb46-5623-4e41-be1a-cfa2c541bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:30.231642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145deb46-5623-4e41-be1a-cfa2c541bc6f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:30.231921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145deb46-5623-4e41-be1a-cfa2c541bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:30.232189 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145deb46-5623-4e41-be1a-cfa2c541bc6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:30.247190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145deb46-5623-4e41-be1a-cfa2c541bc6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:30.248649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145deb46-5623-4e41-be1a-cfa2c541bc6f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-145deb46-5623-4e41-be1a-cfa2c541bc6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:30.265882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:30.266874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:30.267849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:30.267849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:30.267849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:30.267849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:30.272391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 406/816] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:30.273117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145deb46-5623-4e41-be1a-cfa2c541bc6f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:30.324762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:30.353032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ff2ae-cd36-4c62-9c5d-3e64bcc14e22 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:30.390160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59d0f4-6d51-40fb-a2b6-c67d3bfe185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:30.390789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d0f4-6d51-40fb-a2b6-c67d3bfe185c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:30.391065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d0f4-6d51-40fb-a2b6-c67d3bfe185c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:30.391335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d0f4-6d51-40fb-a2b6-c67d3bfe185c None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:30.392127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59d0f4-6d51-40fb-a2b6-c67d3bfe185c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:30.392431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59d0f4-6d51-40fb-a2b6-c67d3bfe185c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:30.394965 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59d0f4-6d51-40fb-a2b6-c67d3bfe185c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:30.396441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145deb46-5623-4e41-be1a-cfa2c541bc6f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:30.420472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 411/817] 10.222.0.73 () {56 vars in 1135 bytes} [Thu Aug 4 21:22:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:30.420472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59d0f4-6d51-40fb-a2b6-c67d3bfe185c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:30.447418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcba1c6-cf04-4b7a-b8ee-f1b59d2b154e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:30.448443 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcba1c6-cf04-4b7a-b8ee-f1b59d2b154e None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:30.448919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcba1c6-cf04-4b7a-b8ee-f1b59d2b154e None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:30.449378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcba1c6-cf04-4b7a-b8ee-f1b59d2b154e None admin] PATH_INFO: `/v3/services/network` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:30.450518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcba1c6-cf04-4b7a-b8ee-f1b59d2b154e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:30.452887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcba1c6-cf04-4b7a-b8ee-f1b59d2b154e None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-8fcba1c6-cf04-4b7a-b8ee-f1b59d2b154e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:30.463918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:30.464966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:30.465846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 407/818] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:30 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:30.467677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcba1c6-cf04-4b7a-b8ee-f1b59d2b154e None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:30.500784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:30.557526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59d0f4-6d51-40fb-a2b6-c67d3bfe185c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:30.586268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcba1c6-cf04-4b7a-b8ee-f1b59d2b154e None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:30.608157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66e80b-ad5a-4486-afc4-881d91ffc14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:30.609416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66e80b-ad5a-4486-afc4-881d91ffc14a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:30.609870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66e80b-ad5a-4486-afc4-881d91ffc14a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:30.610343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66e80b-ad5a-4486-afc4-881d91ffc14a None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:30.612156 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66e80b-ad5a-4486-afc4-881d91ffc14a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:30.612469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66e80b-ad5a-4486-afc4-881d91ffc14a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:30.614018 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66e80b-ad5a-4486-afc4-881d91ffc14a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:30.616809 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:30.620042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 412/819] 10.222.0.73 () {56 vars in 1124 bytes} [Thu Aug 4 21:22:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:30.623016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66e80b-ad5a-4486-afc4-881d91ffc14a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:30.628071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb5deb-560e-4a2f-aec2-06e158e3dbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:30.628315 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5deb-560e-4a2f-aec2-06e158e3dbdb None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:30.628701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5deb-560e-4a2f-aec2-06e158e3dbdb None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:30.628980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5deb-560e-4a2f-aec2-06e158e3dbdb None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:30.635384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb5deb-560e-4a2f-aec2-06e158e3dbdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:30.635384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb5deb-560e-4a2f-aec2-06e158e3dbdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:30.635384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb5deb-560e-4a2f-aec2-06e158e3dbdb None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:30.645345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 408/820] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:30.646210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb5deb-560e-4a2f-aec2-06e158e3dbdb None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:30.751313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66e80b-ad5a-4486-afc4-881d91ffc14a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:30.761121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:30.773489 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb5deb-560e-4a2f-aec2-06e158e3dbdb None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:30.788575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047b58c-100f-46ef-8173-54934022ed8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:30.789127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047b58c-100f-46ef-8173-54934022ed8a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:30.789414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047b58c-100f-46ef-8173-54934022ed8a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:30.789693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047b58c-100f-46ef-8173-54934022ed8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:30.803453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047b58c-100f-46ef-8173-54934022ed8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:30.804927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047b58c-100f-46ef-8173-54934022ed8a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:30.815701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466f817-7759-44ae-8516-b870504890b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:30.816722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f817-7759-44ae-8516-b870504890b8 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:30.817203 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f817-7759-44ae-8516-b870504890b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:30.817768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f817-7759-44ae-8516-b870504890b8 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:30.818966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466f817-7759-44ae-8516-b870504890b8 None admin] RBAC: Adding query filter params (name=network) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:30.819530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466f817-7759-44ae-8516-b870504890b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-1047b58c-100f-46ef-8173-54934022ed8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:30.820498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:30.821370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:30.822300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:30.822300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:30.822300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:30.822300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:30.822300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 413/821] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:30.822968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047b58c-100f-46ef-8173-54934022ed8a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:30.826384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466f817-7759-44ae-8516-b870504890b8 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:30.832696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 409/822] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:30.833717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466f817-7759-44ae-8516-b870504890b8 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:30.887148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:30.942235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047b58c-100f-46ef-8173-54934022ed8a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:30.947335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466f817-7759-44ae-8516-b870504890b8 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:30.976441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20bd21-fe8a-4abe-bda1-6d1a889d9772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:30.977060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20bd21-fe8a-4abe-bda1-6d1a889d9772 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:30.977343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20bd21-fe8a-4abe-bda1-6d1a889d9772 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:30.977613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20bd21-fe8a-4abe-bda1-6d1a889d9772 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:30.978346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20bd21-fe8a-4abe-bda1-6d1a889d9772 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:30.979701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20bd21-fe8a-4abe-bda1-6d1a889d9772 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:30.985564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb6716-e857-4eb2-bb77-6e7178971782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:30.986926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb6716-e857-4eb2-bb77-6e7178971782 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:30.987376 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb6716-e857-4eb2-bb77-6e7178971782 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:30.987789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb6716-e857-4eb2-bb77-6e7178971782 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:30.995857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb6716-e857-4eb2-bb77-6e7178971782 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:30.997535 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb6716-e857-4eb2-bb77-6e7178971782 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-95fb6716-e857-4eb2-bb77-6e7178971782 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:31.010114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:31.014263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:31.015355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:31.015355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:31.015355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:31.015355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:31.015355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:31.015355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:31.015355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:31.015355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:31.015355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 410/823] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:31.015355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb6716-e857-4eb2-bb77-6e7178971782 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:31.032491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:31.101662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 414/824] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:22:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:22:31.102439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20bd21-fe8a-4abe-bda1-6d1a889d9772 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:31.151102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb6716-e857-4eb2-bb77-6e7178971782 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:31.207855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e6940-e764-4b49-9314-9b24dea2bf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:31.208484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e6940-e764-4b49-9314-9b24dea2bf5c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:31.208831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e6940-e764-4b49-9314-9b24dea2bf5c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:31.209174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e6940-e764-4b49-9314-9b24dea2bf5c None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:31.210665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e6940-e764-4b49-9314-9b24dea2bf5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:31.210852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e6940-e764-4b49-9314-9b24dea2bf5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:31.212689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e6940-e764-4b49-9314-9b24dea2bf5c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:31.227634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 411/825] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:31.228575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e6940-e764-4b49-9314-9b24dea2bf5c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:31.247626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20bd21-fe8a-4abe-bda1-6d1a889d9772 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:31.262577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:31.300073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ed3c9-cccf-488d-88a4-19c8a61f870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:31.301145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ed3c9-cccf-488d-88a4-19c8a61f870c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:31.301690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ed3c9-cccf-488d-88a4-19c8a61f870c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:31.301973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ed3c9-cccf-488d-88a4-19c8a61f870c None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:31.302923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ed3c9-cccf-488d-88a4-19c8a61f870c None admin] RBAC: Adding query filter params (type=network) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:31.303289 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ed3c9-cccf-488d-88a4-19c8a61f870c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:31.305344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ed3c9-cccf-488d-88a4-19c8a61f870c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:31.312747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 415/826] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:31.313406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ed3c9-cccf-488d-88a4-19c8a61f870c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:31.315255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:31.350831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e6940-e764-4b49-9314-9b24dea2bf5c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:31.387909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb20e77-8be3-4c9c-955d-83c63056190e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:31.388877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb20e77-8be3-4c9c-955d-83c63056190e None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:31.389377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb20e77-8be3-4c9c-955d-83c63056190e None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:31.389843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb20e77-8be3-4c9c-955d-83c63056190e None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:31.390907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb20e77-8be3-4c9c-955d-83c63056190e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:31.391434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb20e77-8be3-4c9c-955d-83c63056190e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:31.393758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb20e77-8be3-4c9c-955d-83c63056190e None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:31.408889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 412/827] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:31.410380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb20e77-8be3-4c9c-955d-83c63056190e None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:31.442608 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ed3c9-cccf-488d-88a4-19c8a61f870c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:31.481033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c95188-5ae4-4f77-a5e9-f82b5583bfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:31.481966 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c95188-5ae4-4f77-a5e9-f82b5583bfa7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:31.482451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c95188-5ae4-4f77-a5e9-f82b5583bfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:31.482982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c95188-5ae4-4f77-a5e9-f82b5583bfa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:31.491261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c95188-5ae4-4f77-a5e9-f82b5583bfa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:31.492919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c95188-5ae4-4f77-a5e9-f82b5583bfa7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-61c95188-5ae4-4f77-a5e9-f82b5583bfa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:31.505425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:31.506402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:31.507305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:31.507305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:31.507305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:31.507305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:31.507305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:31.507305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:31.507305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:31.507305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:31.509676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 416/828] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:31.510738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c95188-5ae4-4f77-a5e9-f82b5583bfa7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:31.538273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb20e77-8be3-4c9c-955d-83c63056190e None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:31.554999 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:31.569464 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:31.576153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc0c78-f63c-41cf-a436-f6b5ac662709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:31.576819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc0c78-f63c-41cf-a436-f6b5ac662709 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:31.577099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc0c78-f63c-41cf-a436-f6b5ac662709 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:31.577415 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc0c78-f63c-41cf-a436-f6b5ac662709 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:31.578158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc0c78-f63c-41cf-a436-f6b5ac662709 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:31.579720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc0c78-f63c-41cf-a436-f6b5ac662709 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:31.636663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c95188-5ae4-4f77-a5e9-f82b5583bfa7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:31.671685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb8ebd-2898-4bc9-a718-79c3b880696d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:31.672223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb8ebd-2898-4bc9-a718-79c3b880696d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:31.672505 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb8ebd-2898-4bc9-a718-79c3b880696d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:31.672779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb8ebd-2898-4bc9-a718-79c3b880696d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:31.673563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb8ebd-2898-4bc9-a718-79c3b880696d None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=e7b8267780364a2181abf5817b54ce5c) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:31.673874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb8ebd-2898-4bc9-a718-79c3b880696d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:31.675973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb8ebd-2898-4bc9-a718-79c3b880696d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:31.686060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 417/829] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:31 2022] GET /identity/v3/role_assignments?user.id=e7b8267780364a2181abf5817b54ce5c&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:31.686060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb8ebd-2898-4bc9-a718-79c3b880696d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:31.693053 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 413/830] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:22:31 2022] POST /identity/v3/endpoints => generated 337 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:22:31.777750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c4fda-3c14-4a3a-ae27-573e2741aa4b None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:31.778106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c4fda-3c14-4a3a-ae27-573e2741aa4b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:31.778337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c4fda-3c14-4a3a-ae27-573e2741aa4b None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:31.780078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 414/831] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:22:31.789968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba591-0be1-49c0-9047-ef1e48809e33 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:31.790327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba591-0be1-49c0-9047-ef1e48809e33 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:31.790566 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba591-0be1-49c0-9047-ef1e48809e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:31.822777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c6aba591-0be1-49c0-9047-ef1e48809e33 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:31.826709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c6aba591-0be1-49c0-9047-ef1e48809e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:31.865608 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:31.874319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb8ebd-2898-4bc9-a718-79c3b880696d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:31.932426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f1272-3bab-4c28-a277-333201f7b95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:31.933061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f1272-3bab-4c28-a277-333201f7b95b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:31.933346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f1272-3bab-4c28-a277-333201f7b95b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:31.933744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f1272-3bab-4c28-a277-333201f7b95b None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:31.934746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f1272-3bab-4c28-a277-333201f7b95b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:31.935296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f1272-3bab-4c28-a277-333201f7b95b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:31.936621 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f1272-3bab-4c28-a277-333201f7b95b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:31.946231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 415/832] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:31.951654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 418/833] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:31.957696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba591-0be1-49c0-9047-ef1e48809e33 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:31.977344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f1272-3bab-4c28-a277-333201f7b95b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:32.093343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba591-0be1-49c0-9047-ef1e48809e33 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:32.109177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f1272-3bab-4c28-a277-333201f7b95b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:32.118201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:32.142366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118231ca-7572-42a8-8ab6-347796885d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:32.144026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118231ca-7572-42a8-8ab6-347796885d6a None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:32.144456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118231ca-7572-42a8-8ab6-347796885d6a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:32.144738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118231ca-7572-42a8-8ab6-347796885d6a None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/3342e1039131496ba121b04685ec7b3d/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:32.144843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03feff7c-267c-4c62-b743-aca1783bcfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:32.145479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feff7c-267c-4c62-b743-aca1783bcfce None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:32.145763 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feff7c-267c-4c62-b743-aca1783bcfce None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:32.146082 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feff7c-267c-4c62-b743-aca1783bcfce None admin] PATH_INFO: `/v3/roles/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:32.146892 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118231ca-7572-42a8-8ab6-347796885d6a None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=3342e1039131496ba121b04685ec7b3d, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:32.148559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118231ca-7572-42a8-8ab6-347796885d6a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:32.156735 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03feff7c-267c-4c62-b743-aca1783bcfce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-03feff7c-267c-4c62-b743-aca1783bcfce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:32.162987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:32.164787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:32.165630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 419/834] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:22:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:32.170282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03feff7c-267c-4c62-b743-aca1783bcfce None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:32.185864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:32.217176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 416/835] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:22:31 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/3342e1039131496ba121b04685ec7b3d/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:22:32.296013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03feff7c-267c-4c62-b743-aca1783bcfce None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:32.354748 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978e197-c73f-4ddf-b7b0-3cc5f553c49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:32.355806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978e197-c73f-4ddf-b7b0-3cc5f553c49d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:32.356407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978e197-c73f-4ddf-b7b0-3cc5f553c49d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:32.356902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978e197-c73f-4ddf-b7b0-3cc5f553c49d None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:32.357861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978e197-c73f-4ddf-b7b0-3cc5f553c49d None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:32.358426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978e197-c73f-4ddf-b7b0-3cc5f553c49d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:32.360896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978e197-c73f-4ddf-b7b0-3cc5f553c49d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:32.373732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 420/836] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:22:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:32.381440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118231ca-7572-42a8-8ab6-347796885d6a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:32.492294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118231ca-7572-42a8-8ab6-347796885d6a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:32.533122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0d909-cf55-4bec-bf33-9f6e520760a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:32.534383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0d909-cf55-4bec-bf33-9f6e520760a1 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:32.534590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0d909-cf55-4bec-bf33-9f6e520760a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:32.534868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0d909-cf55-4bec-bf33-9f6e520760a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:32.545927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0d909-cf55-4bec-bf33-9f6e520760a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:32.547413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0d909-cf55-4bec-bf33-9f6e520760a1 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-93d0d909-cf55-4bec-bf33-9f6e520760a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:32.563262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:32.564397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:32.566192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:32.566192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:32.566192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:32.566192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:32.566192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 417/837] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:32.572162 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978e197-c73f-4ddf-b7b0-3cc5f553c49d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:32.658709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:32.685470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978e197-c73f-4ddf-b7b0-3cc5f553c49d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:32.723913 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf9ba4-69ba-43f0-bed2-830eeca34ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:32.724594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9ba4-69ba-43f0-bed2-830eeca34ea7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:32.724937 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9ba4-69ba-43f0-bed2-830eeca34ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:32.725221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9ba4-69ba-43f0-bed2-830eeca34ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:32.725909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf9ba4-69ba-43f0-bed2-830eeca34ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:32.726247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf9ba4-69ba-43f0-bed2-830eeca34ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:32.727605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf9ba4-69ba-43f0-bed2-830eeca34ea7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:32.740374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 421/838] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:32.745879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0d909-cf55-4bec-bf33-9f6e520760a1 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:32.776548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:32.850852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0d909-cf55-4bec-bf33-9f6e520760a1 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:32.887790 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfec8ee-fa44-4be9-9fbf-4368735d3359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:32.888447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfec8ee-fa44-4be9-9fbf-4368735d3359 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:32.888732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfec8ee-fa44-4be9-9fbf-4368735d3359 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:32.889008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfec8ee-fa44-4be9-9fbf-4368735d3359 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:32.897133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfec8ee-fa44-4be9-9fbf-4368735d3359 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:32.898619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfec8ee-fa44-4be9-9fbf-4368735d3359 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-2bfec8ee-fa44-4be9-9fbf-4368735d3359 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:32.910900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:32.911893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:32.912866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:32.912866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:32.912866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:32.912866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:32.912866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:32.912866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:32.912866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:32.912866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:32.912866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:32.912866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:32.912866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 418/839] 10.222.0.73 () {56 vars in 1117 bytes} [Thu Aug 4 21:22:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:32.919743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf9ba4-69ba-43f0-bed2-830eeca34ea7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:33.035817 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf9ba4-69ba-43f0-bed2-830eeca34ea7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:33.074249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85694068-cd79-4c0a-aca8-bd67dad94115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:33.074900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85694068-cd79-4c0a-aca8-bd67dad94115 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:33.075179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85694068-cd79-4c0a-aca8-bd67dad94115 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:33.075459 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85694068-cd79-4c0a-aca8-bd67dad94115 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:33.076288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85694068-cd79-4c0a-aca8-bd67dad94115 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:33.076602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85694068-cd79-4c0a-aca8-bd67dad94115 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:33.078133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85694068-cd79-4c0a-aca8-bd67dad94115 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:33.080669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:33.100490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 422/840] 10.222.0.73 () {56 vars in 1149 bytes} [Thu Aug 4 21:22:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:33.106112 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfec8ee-fa44-4be9-9fbf-4368735d3359 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:33.207615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:33.224266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfec8ee-fa44-4be9-9fbf-4368735d3359 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:33.262081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd57ea-d935-4f7c-8709-ff37b3b26141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:33.262704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd57ea-d935-4f7c-8709-ff37b3b26141 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:33.263004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd57ea-d935-4f7c-8709-ff37b3b26141 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:33.263302 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd57ea-d935-4f7c-8709-ff37b3b26141 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:33.308902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cef999-75c7-465d-914f-459b1476fb93 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:33.309184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cef999-75c7-465d-914f-459b1476fb93 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:33.309367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cef999-75c7-465d-914f-459b1476fb93 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:33.311934 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 423/841] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:22:33.316388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b8221-6033-42f4-a4ee-389374c67319 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:33.316675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b8221-6033-42f4-a4ee-389374c67319 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:33.316952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b8221-6033-42f4-a4ee-389374c67319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:33.334038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd57ea-d935-4f7c-8709-ff37b3b26141 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:33.335512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd57ea-d935-4f7c-8709-ff37b3b26141 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:33.337371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-521b8221-6033-42f4-a4ee-389374c67319 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:33.339447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-521b8221-6033-42f4-a4ee-389374c67319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:33.433745 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:33.434113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 424/842] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:33.451964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 419/843] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:33 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:22:33.457350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b8221-6033-42f4-a4ee-389374c67319 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:33.462357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd57ea-d935-4f7c-8709-ff37b3b26141 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:33.588453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b8221-6033-42f4-a4ee-389374c67319 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:33.590056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd57ea-d935-4f7c-8709-ff37b3b26141 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:33.614774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:33.627193 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95dac8-bd49-41a6-91f7-99293f233ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:33.627576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5b328-ef64-45f8-a196-f0377358c731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:33.642244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5b328-ef64-45f8-a196-f0377358c731 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:33.642387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95dac8-bd49-41a6-91f7-99293f233ebc None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:33.642467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5b328-ef64-45f8-a196-f0377358c731 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:33.642537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95dac8-bd49-41a6-91f7-99293f233ebc None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:33.643126 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5b328-ef64-45f8-a196-f0377358c731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:33.643208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95dac8-bd49-41a6-91f7-99293f233ebc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:33.643208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95dac8-bd49-41a6-91f7-99293f233ebc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:33.643208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95dac8-bd49-41a6-91f7-99293f233ebc None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:33.643402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5b328-ef64-45f8-a196-f0377358c731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-1c95dac8-bd49-41a6-91f7-99293f233ebc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:33.643476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:33.644439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:22:33.645284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:33.646055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5b328-ef64-45f8-a196-f0377358c731 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:33.650110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 425/844] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:22:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:33.652941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95dac8-bd49-41a6-91f7-99293f233ebc None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-fab5b328-ef64-45f8-a196-f0377358c731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:33.657378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:33.658607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:33.659719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:33.659719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:33.659719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:33.659719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:33.663532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 420/845] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:33.664590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5b328-ef64-45f8-a196-f0377358c731 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:33.711540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:33.762630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95dac8-bd49-41a6-91f7-99293f233ebc None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:33.773475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5b328-ef64-45f8-a196-f0377358c731 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:33.799550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63163d-b072-4655-bb00-99e442930e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:33.800497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63163d-b072-4655-bb00-99e442930e47 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:33.800969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63163d-b072-4655-bb00-99e442930e47 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:33.801430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63163d-b072-4655-bb00-99e442930e47 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:33.802318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63163d-b072-4655-bb00-99e442930e47 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:33.802798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63163d-b072-4655-bb00-99e442930e47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:33.804380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63163d-b072-4655-bb00-99e442930e47 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:33.814114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06160a7f-cff6-4f9d-b3b2-fd43bba87f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:33.814114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06160a7f-cff6-4f9d-b3b2-fd43bba87f48 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:33.814114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06160a7f-cff6-4f9d-b3b2-fd43bba87f48 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:33.814114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06160a7f-cff6-4f9d-b3b2-fd43bba87f48 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:33.814578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06160a7f-cff6-4f9d-b3b2-fd43bba87f48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:33.814578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06160a7f-cff6-4f9d-b3b2-fd43bba87f48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:33.814578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06160a7f-cff6-4f9d-b3b2-fd43bba87f48 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:33.814796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 426/846] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:22:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:33.816166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63163d-b072-4655-bb00-99e442930e47 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:33.833697 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 421/847] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:33.834738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06160a7f-cff6-4f9d-b3b2-fd43bba87f48 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:33.873469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:33.930789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63163d-b072-4655-bb00-99e442930e47 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:33.946626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:33.950313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06160a7f-cff6-4f9d-b3b2-fd43bba87f48 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:33.968068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaef76c-dcf3-4d1e-9ba4-dd829614e629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:33.969109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaef76c-dcf3-4d1e-9ba4-dd829614e629 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:33.969719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaef76c-dcf3-4d1e-9ba4-dd829614e629 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:33.970185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaef76c-dcf3-4d1e-9ba4-dd829614e629 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:33.972071 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaef76c-dcf3-4d1e-9ba4-dd829614e629 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:33.972667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaef76c-dcf3-4d1e-9ba4-dd829614e629 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:33.974884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaef76c-dcf3-4d1e-9ba4-dd829614e629 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:33.989943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 427/848] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:22:33 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:33.995557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa927ebb-a64a-40f1-a456-38dd3458aed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:33.996244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa927ebb-a64a-40f1-a456-38dd3458aed2 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:33.996695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa927ebb-a64a-40f1-a456-38dd3458aed2 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:33.997273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa927ebb-a64a-40f1-a456-38dd3458aed2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:34.010577 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa927ebb-a64a-40f1-a456-38dd3458aed2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:34.012295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa927ebb-a64a-40f1-a456-38dd3458aed2 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-fa927ebb-a64a-40f1-a456-38dd3458aed2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:34.027777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:34.028750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:34.030206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:34.030206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:34.030206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:34.030206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:34.030206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:34.030206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:34.030206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:34.030206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:34.030206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 422/849] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:34.035317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaef76c-dcf3-4d1e-9ba4-dd829614e629 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:34.091198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169384c-9e7b-422a-a81d-9f59c3374d8d None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:34.091559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169384c-9e7b-422a-a81d-9f59c3374d8d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:34.091756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169384c-9e7b-422a-a81d-9f59c3374d8d None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:34.093540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 423/850] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:34.100327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ed4d7-34cd-44fc-9c1d-4aa8684d59a0 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:34.100651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ed4d7-34cd-44fc-9c1d-4aa8684d59a0 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:34.100884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ed4d7-34cd-44fc-9c1d-4aa8684d59a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:34.130356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d92ed4d7-34cd-44fc-9c1d-4aa8684d59a0 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:34.131510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d92ed4d7-34cd-44fc-9c1d-4aa8684d59a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:34.181210 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaef76c-dcf3-4d1e-9ba4-dd829614e629 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:34.200721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:34.224595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f507e9-4d34-4d8a-ac44-5cc20c9f8cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:34.225517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f507e9-4d34-4d8a-ac44-5cc20c9f8cf0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:34.225797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f507e9-4d34-4d8a-ac44-5cc20c9f8cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:34.226102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f507e9-4d34-4d8a-ac44-5cc20c9f8cf0 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:34.226867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f507e9-4d34-4d8a-ac44-5cc20c9f8cf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:34.227194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f507e9-4d34-4d8a-ac44-5cc20c9f8cf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:34.228698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f507e9-4d34-4d8a-ac44-5cc20c9f8cf0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:34.244982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 428/851] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:34.245790 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 424/852] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:34.252947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f507e9-4d34-4d8a-ac44-5cc20c9f8cf0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:34.272439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ed4d7-34cd-44fc-9c1d-4aa8684d59a0 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:34.382833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:34.397592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ed4d7-34cd-44fc-9c1d-4aa8684d59a0 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:34.411615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f507e9-4d34-4d8a-ac44-5cc20c9f8cf0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:34.463157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22015e45-9ce7-43df-9948-0fc5e16ab067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:34.463920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22015e45-9ce7-43df-9948-0fc5e16ab067 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:34.464204 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22015e45-9ce7-43df-9948-0fc5e16ab067 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:34.464484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22015e45-9ce7-43df-9948-0fc5e16ab067 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:34.465150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22015e45-9ce7-43df-9948-0fc5e16ab067 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:34.466800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22015e45-9ce7-43df-9948-0fc5e16ab067 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:34.474438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff44245-0244-41f0-b312-2137bb90176f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:34.475234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff44245-0244-41f0-b312-2137bb90176f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:34.475654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff44245-0244-41f0-b312-2137bb90176f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:34.476057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff44245-0244-41f0-b312-2137bb90176f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:34.476996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff44245-0244-41f0-b312-2137bb90176f None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=6027cfb8625c4d028e7160a85e81bb74) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:34.477448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff44245-0244-41f0-b312-2137bb90176f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-22015e45-9ce7-43df-9948-0fc5e16ab067 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:34.477922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:34.480055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:34.481965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 425/853] 10.222.0.73 () {56 vars in 1132 bytes} [Thu Aug 4 21:22:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:34.485097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746a541-868d-41cd-837b-5466d84457ad None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:34.485097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746a541-868d-41cd-837b-5466d84457ad None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:34.485097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746a541-868d-41cd-837b-5466d84457ad None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:34.486951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 426/854] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:34.488101 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff44245-0244-41f0-b312-2137bb90176f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:34.489719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436483f-cabe-4f68-93be-c2285020b30f None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:34.490561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436483f-cabe-4f68-93be-c2285020b30f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:34.491236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436483f-cabe-4f68-93be-c2285020b30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:34.499096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 429/855] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:34 2022] GET /identity/v3/role_assignments?user.id=6027cfb8625c4d028e7160a85e81bb74&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:34.500724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff44245-0244-41f0-b312-2137bb90176f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:34.527334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0436483f-cabe-4f68-93be-c2285020b30f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:34.529825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0436483f-cabe-4f68-93be-c2285020b30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:34.594372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:34.660097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 427/856] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:34.684402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436483f-cabe-4f68-93be-c2285020b30f None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:34.684629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff44245-0244-41f0-b312-2137bb90176f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:34.739174 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acd1f3-d4a9-45f6-85d5-27b850255070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:34.740102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acd1f3-d4a9-45f6-85d5-27b850255070 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:34.740383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acd1f3-d4a9-45f6-85d5-27b850255070 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:34.740651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acd1f3-d4a9-45f6-85d5-27b850255070 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:34.742395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acd1f3-d4a9-45f6-85d5-27b850255070 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:34.742720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acd1f3-d4a9-45f6-85d5-27b850255070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:34.744130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acd1f3-d4a9-45f6-85d5-27b850255070 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:34.749758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 430/857] 10.222.0.73 () {56 vars in 1126 bytes} [Thu Aug 4 21:22:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:34.756770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acd1f3-d4a9-45f6-85d5-27b850255070 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:34.817876 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:34.839414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436483f-cabe-4f68-93be-c2285020b30f None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:34.922535 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c97fe-03bb-4cda-8618-eb5f03ee5ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:34.922535 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c97fe-03bb-4cda-8618-eb5f03ee5ea8 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:34.922535 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c97fe-03bb-4cda-8618-eb5f03ee5ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:34.922535 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c97fe-03bb-4cda-8618-eb5f03ee5ea8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:34.922893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c97fe-03bb-4cda-8618-eb5f03ee5ea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:34.922963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acd1f3-d4a9-45f6-85d5-27b850255070 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-469c97fe-03bb-4cda-8618-eb5f03ee5ea8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:34.923048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:34.924075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:22:34.925000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:34.925652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:34.925721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 428/858] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:34.925721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c97fe-03bb-4cda-8618-eb5f03ee5ea8 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:34.957660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb35c6-eb1d-4cc9-af17-79c171ff8824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:34.958802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb35c6-eb1d-4cc9-af17-79c171ff8824 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:34.959555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb35c6-eb1d-4cc9-af17-79c171ff8824 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:34.960328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb35c6-eb1d-4cc9-af17-79c171ff8824 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:34.961477 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb35c6-eb1d-4cc9-af17-79c171ff8824 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:34.962060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb35c6-eb1d-4cc9-af17-79c171ff8824 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:34.963707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb35c6-eb1d-4cc9-af17-79c171ff8824 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:34.970519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 431/859] 10.222.0.73 () {56 vars in 1126 bytes} [Thu Aug 4 21:22:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:35.057391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c97fe-03bb-4cda-8618-eb5f03ee5ea8 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:35.106091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f730ba-5a39-4222-afb8-fde74d04ab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:35.107570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f730ba-5a39-4222-afb8-fde74d04ab43 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:35.108272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f730ba-5a39-4222-afb8-fde74d04ab43 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:35.108997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f730ba-5a39-4222-afb8-fde74d04ab43 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:35.110529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f730ba-5a39-4222-afb8-fde74d04ab43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:35.112156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f730ba-5a39-4222-afb8-fde74d04ab43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:35.115254 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f730ba-5a39-4222-afb8-fde74d04ab43 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:35.131947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 429/860] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:22:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:35.139667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb35c6-eb1d-4cc9-af17-79c171ff8824 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:35.271888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb35c6-eb1d-4cc9-af17-79c171ff8824 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:35.312819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe83de-3858-40ab-8ed5-5a9e65d9ff4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:35.314115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe83de-3858-40ab-8ed5-5a9e65d9ff4a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:35.314744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe83de-3858-40ab-8ed5-5a9e65d9ff4a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:35.315289 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe83de-3858-40ab-8ed5-5a9e65d9ff4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:35.327424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe83de-3858-40ab-8ed5-5a9e65d9ff4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:35.329192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe83de-3858-40ab-8ed5-5a9e65d9ff4a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-7cbe83de-3858-40ab-8ed5-5a9e65d9ff4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:35.346246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:35.347247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:35.348137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:35.348137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:35.348137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:35.348137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:35.350391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 432/861] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:35.359187 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f730ba-5a39-4222-afb8-fde74d04ab43 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:35.374074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:35.504068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f730ba-5a39-4222-afb8-fde74d04ab43 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:35.540749 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41803e-b05f-444c-94a3-8fad4327d70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:35.541374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41803e-b05f-444c-94a3-8fad4327d70a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:35.541650 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41803e-b05f-444c-94a3-8fad4327d70a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:35.541921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41803e-b05f-444c-94a3-8fad4327d70a None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:35.542721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41803e-b05f-444c-94a3-8fad4327d70a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:35.543040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41803e-b05f-444c-94a3-8fad4327d70a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:35.544394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41803e-b05f-444c-94a3-8fad4327d70a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:35.557508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 430/862] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:35.563228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe83de-3858-40ab-8ed5-5a9e65d9ff4a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:35.587072 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:35.703512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe83de-3858-40ab-8ed5-5a9e65d9ff4a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:35.756596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caa8bf-72b7-45e1-b154-9ab59fd71e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:35.757579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caa8bf-72b7-45e1-b154-9ab59fd71e3a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:35.758000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caa8bf-72b7-45e1-b154-9ab59fd71e3a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:35.758445 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caa8bf-72b7-45e1-b154-9ab59fd71e3a None admin] PATH_INFO: `/v3/users/swift` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:35.770404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9caa8bf-72b7-45e1-b154-9ab59fd71e3a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:35.772829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9caa8bf-72b7-45e1-b154-9ab59fd71e3a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-e9caa8bf-72b7-45e1-b154-9ab59fd71e3a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:35.793237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:35.794317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:35.795310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:35.795310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:35.795310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:35.795310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:35.795310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:35.795310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:35.795310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:35.795310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:35.795310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 21:22:35.795310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:35.795878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 433/863] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:22:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:35.801860 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41803e-b05f-444c-94a3-8fad4327d70a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:35.874354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:35.911584 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41803e-b05f-444c-94a3-8fad4327d70a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:35.956551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793fc9a-816e-45b1-a2ea-a4b94e87dfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:35.957179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793fc9a-816e-45b1-a2ea-a4b94e87dfef None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:35.957456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793fc9a-816e-45b1-a2ea-a4b94e87dfef None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:35.957727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793fc9a-816e-45b1-a2ea-a4b94e87dfef None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:35.958566 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793fc9a-816e-45b1-a2ea-a4b94e87dfef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:35.958909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793fc9a-816e-45b1-a2ea-a4b94e87dfef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:35.960367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793fc9a-816e-45b1-a2ea-a4b94e87dfef None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:35.980205 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 431/864] 10.222.0.73 () {56 vars in 1135 bytes} [Thu Aug 4 21:22:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:35.986024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caa8bf-72b7-45e1-b154-9ab59fd71e3a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:36.078117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:36.093793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caa8bf-72b7-45e1-b154-9ab59fd71e3a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:36.131720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b02b7-269f-43e8-a20a-76f6d271384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:36.132343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b02b7-269f-43e8-a20a-76f6d271384c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:36.132620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b02b7-269f-43e8-a20a-76f6d271384c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:36.132891 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b02b7-269f-43e8-a20a-76f6d271384c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:36.204826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b02b7-269f-43e8-a20a-76f6d271384c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:36.206385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b02b7-269f-43e8-a20a-76f6d271384c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:36.264472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632d16-3d05-4225-aa4d-9d5f39e50e9a None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:36.265213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632d16-3d05-4225-aa4d-9d5f39e50e9a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:36.265213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632d16-3d05-4225-aa4d-9d5f39e50e9a None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:36.267147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 432/865] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:36.278377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36027439-a319-45af-9954-880fc974eb18 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:36.279024 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36027439-a319-45af-9954-880fc974eb18 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:36.279527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36027439-a319-45af-9954-880fc974eb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:36.301560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-36027439-a319-45af-9954-880fc974eb18 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:36.303698 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-36027439-a319-45af-9954-880fc974eb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:36.306401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:36.334363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 434/866] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:22:36.340957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b02b7-269f-43e8-a20a-76f6d271384c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:36.398647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 433/867] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:36.424176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36027439-a319-45af-9954-880fc974eb18 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:36.480822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b02b7-269f-43e8-a20a-76f6d271384c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:36.485350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:36.531219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b830b62-a320-438d-ad13-09579091b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:36.531932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b830b62-a320-438d-ad13-09579091b4dd None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:36.532217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b830b62-a320-438d-ad13-09579091b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:36.532497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b830b62-a320-438d-ad13-09579091b4dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:36.545423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36027439-a319-45af-9954-880fc974eb18 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:36.546969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b830b62-a320-438d-ad13-09579091b4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:36.550665 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b830b62-a320-438d-ad13-09579091b4dd None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-4b830b62-a320-438d-ad13-09579091b4dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:36.573316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:36.574324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:36.575274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:36.575274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:36.575274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:36.575274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:36.578087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 435/868] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:36.580266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b830b62-a320-438d-ad13-09579091b4dd None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:36.599548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c6bba-ec5e-4a92-8113-0796e2866a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:36.600842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c6bba-ec5e-4a92-8113-0796e2866a84 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:36.601331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c6bba-ec5e-4a92-8113-0796e2866a84 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:36.602076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c6bba-ec5e-4a92-8113-0796e2866a84 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:36.603712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8c6bba-ec5e-4a92-8113-0796e2866a84 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:36.605623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8c6bba-ec5e-4a92-8113-0796e2866a84 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:36.627751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:36.732994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 434/869] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:22:36 2022] POST /identity/v3/services => generated 251 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:22:36.753563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b830b62-a320-438d-ad13-09579091b4dd None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:36.807374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca05487-745e-40df-a3a8-8185cf26e68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:36.808440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05487-745e-40df-a3a8-8185cf26e68b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:36.808907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05487-745e-40df-a3a8-8185cf26e68b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:36.809394 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05487-745e-40df-a3a8-8185cf26e68b None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:36.810400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca05487-745e-40df-a3a8-8185cf26e68b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:36.811054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca05487-745e-40df-a3a8-8185cf26e68b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:36.812709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca05487-745e-40df-a3a8-8185cf26e68b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:36.829278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 436/870] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:36.835243 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c6bba-ec5e-4a92-8113-0796e2866a84 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:36.934689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2909d08-6085-4198-8677-2d93044c0b26 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:36.934980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2909d08-6085-4198-8677-2d93044c0b26 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:36.935259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2909d08-6085-4198-8677-2d93044c0b26 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:36.936623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 437/871] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:36.942590 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073a3d7-7235-45bf-bf3c-03defcda0553 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:36.943150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073a3d7-7235-45bf-bf3c-03defcda0553 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:36.943644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073a3d7-7235-45bf-bf3c-03defcda0553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:36.963468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-e073a3d7-7235-45bf-bf3c-03defcda0553 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:36.964702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e073a3d7-7235-45bf-bf3c-03defcda0553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:36.967987 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c6bba-ec5e-4a92-8113-0796e2866a84 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:36.985758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:37.026956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa84e3-026f-4e65-aee7-bf2f0a1e9bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:37.027737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa84e3-026f-4e65-aee7-bf2f0a1e9bb3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:37.028095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa84e3-026f-4e65-aee7-bf2f0a1e9bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:37.028466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa84e3-026f-4e65-aee7-bf2f0a1e9bb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:37.047255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa84e3-026f-4e65-aee7-bf2f0a1e9bb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:37.048875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa84e3-026f-4e65-aee7-bf2f0a1e9bb3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-72fa84e3-026f-4e65-aee7-bf2f0a1e9bb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:37.072475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:37.073451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:37.074420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:37.074420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:37.074420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:37.074420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:37.074420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:37.074420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:37.074420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:37.074420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:37.086523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 435/872] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:37.087444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa84e3-026f-4e65-aee7-bf2f0a1e9bb3 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:37.101164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 438/873] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:36 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:37.139305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073a3d7-7235-45bf-bf3c-03defcda0553 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:37.162249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:37.279377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa84e3-026f-4e65-aee7-bf2f0a1e9bb3 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:37.309344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073a3d7-7235-45bf-bf3c-03defcda0553 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:37.327774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568563c-6f6f-46d9-8f49-c18731e79abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:37.328408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568563c-6f6f-46d9-8f49-c18731e79abb None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:37.328688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568563c-6f6f-46d9-8f49-c18731e79abb None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:37.328958 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568563c-6f6f-46d9-8f49-c18731e79abb None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:37.329739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568563c-6f6f-46d9-8f49-c18731e79abb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:37.333749 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568563c-6f6f-46d9-8f49-c18731e79abb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:37.334681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568563c-6f6f-46d9-8f49-c18731e79abb None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:37.349408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 436/874] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:37.350778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458177f-9a66-4213-8a94-e2f1f70a0bcb None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:37.351251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458177f-9a66-4213-8a94-e2f1f70a0bcb None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:37.351695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458177f-9a66-4213-8a94-e2f1f70a0bcb None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:37.353174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 437/875] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:37.359801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b7fba-db83-46ce-920a-e9732e712afc None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:37.360104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b7fba-db83-46ce-920a-e9732e712afc None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:37.360362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b7fba-db83-46ce-920a-e9732e712afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:37.360968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3316a61-c52a-441b-9f55-4867b7a41c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:37.362510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3316a61-c52a-441b-9f55-4867b7a41c95 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:37.362786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3316a61-c52a-441b-9f55-4867b7a41c95 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:37.363166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3316a61-c52a-441b-9f55-4867b7a41c95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:37.377181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3316a61-c52a-441b-9f55-4867b7a41c95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:37.379897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-3a5b7fba-db83-46ce-920a-e9732e712afc None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:37.382153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3a5b7fba-db83-46ce-920a-e9732e712afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-b3316a61-c52a-441b-9f55-4867b7a41c95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:37.382599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:37.389384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 439/876] 10.222.0.73 () {56 vars in 1120 bytes} [Thu Aug 4 21:22:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:37.391481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3316a61-c52a-441b-9f55-4867b7a41c95 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:37.429222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:37.483450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 438/877] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:37 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:37.484732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b7fba-db83-46ce-920a-e9732e712afc None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:37.533744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:37.560386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3316a61-c52a-441b-9f55-4867b7a41c95 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:37.600088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65666d7-fd1b-4804-817e-43462059431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:37.600818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65666d7-fd1b-4804-817e-43462059431e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:37.601162 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65666d7-fd1b-4804-817e-43462059431e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:37.601512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65666d7-fd1b-4804-817e-43462059431e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:37.602411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65666d7-fd1b-4804-817e-43462059431e None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=3342e1039131496ba121b04685ec7b3d) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:37.602813 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65666d7-fd1b-4804-817e-43462059431e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:37.604316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65666d7-fd1b-4804-817e-43462059431e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:37.608070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b7fba-db83-46ce-920a-e9732e712afc None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:37.622053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 440/878] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:37 2022] GET /identity/v3/role_assignments?user.id=3342e1039131496ba121b04685ec7b3d&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:37.622053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65666d7-fd1b-4804-817e-43462059431e None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:37.657531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238265c2-28ea-4208-8f6f-25a866581fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:37.658502 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238265c2-28ea-4208-8f6f-25a866581fa7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:37.658983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238265c2-28ea-4208-8f6f-25a866581fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:37.659351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238265c2-28ea-4208-8f6f-25a866581fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:37.660275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238265c2-28ea-4208-8f6f-25a866581fa7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:37.660734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238265c2-28ea-4208-8f6f-25a866581fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:37.662259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238265c2-28ea-4208-8f6f-25a866581fa7 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:37.672179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 439/879] 10.222.0.73 () {56 vars in 1115 bytes} [Thu Aug 4 21:22:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:37.678427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238265c2-28ea-4208-8f6f-25a866581fa7 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:37.713483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:37.768302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65666d7-fd1b-4804-817e-43462059431e None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:37.803786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238265c2-28ea-4208-8f6f-25a866581fa7 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:37.822127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7c9cc-d6ed-479f-bd5c-3bd0cf108612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:37.822764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7c9cc-d6ed-479f-bd5c-3bd0cf108612 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:37.823047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7c9cc-d6ed-479f-bd5c-3bd0cf108612 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:37.823327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7c9cc-d6ed-479f-bd5c-3bd0cf108612 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:37.824098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7c9cc-d6ed-479f-bd5c-3bd0cf108612 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:37.825471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7c9cc-d6ed-479f-bd5c-3bd0cf108612 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:37.844722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c64b7-7fd1-404c-9e7a-e9683f98501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:37.845388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c64b7-7fd1-404c-9e7a-e9683f98501f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:37.847216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c64b7-7fd1-404c-9e7a-e9683f98501f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:37.847216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c64b7-7fd1-404c-9e7a-e9683f98501f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:37.858078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c64b7-7fd1-404c-9e7a-e9683f98501f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:37.859562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c64b7-7fd1-404c-9e7a-e9683f98501f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-b45c64b7-7fd1-404c-9e7a-e9683f98501f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:37.876866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:37.878169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:37.879529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:37.879529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:37.879529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:37.879529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:37.879529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 440/880] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:37.884232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c64b7-7fd1-404c-9e7a-e9683f98501f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:37.942956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:37.950938 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 441/881] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:22:37 2022] POST /identity/v3/services => generated 269 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:22:38.004244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:38.037743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c64b7-7fd1-404c-9e7a-e9683f98501f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:38.089735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816c165-ce7f-4a8f-821a-5742ddd5077e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:38.090803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816c165-ce7f-4a8f-821a-5742ddd5077e None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:38.091357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816c165-ce7f-4a8f-821a-5742ddd5077e None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:38.091904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816c165-ce7f-4a8f-821a-5742ddd5077e None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:38.092878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816c165-ce7f-4a8f-821a-5742ddd5077e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:38.093412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816c165-ce7f-4a8f-821a-5742ddd5077e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:38.095161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816c165-ce7f-4a8f-821a-5742ddd5077e None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:38.112163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 441/882] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:38.120700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7c9cc-d6ed-479f-bd5c-3bd0cf108612 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:38.296402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7c9cc-d6ed-479f-bd5c-3bd0cf108612 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:38.335263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a50d5-05de-4c81-aab6-78a8a5ee5111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:38.336189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a50d5-05de-4c81-aab6-78a8a5ee5111 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:38.336189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a50d5-05de-4c81-aab6-78a8a5ee5111 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:38.336432 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a50d5-05de-4c81-aab6-78a8a5ee5111 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:38.344871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a50d5-05de-4c81-aab6-78a8a5ee5111 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:38.346581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a50d5-05de-4c81-aab6-78a8a5ee5111 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-981a50d5-05de-4c81-aab6-78a8a5ee5111 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:38.358543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:38.360369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:38.362983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:38.362983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:38.362983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:38.362983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:38.362983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:38.362983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:38.362983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:38.362983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:38.362983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:38.362983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:38.362983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 442/883] 10.222.0.73 () {56 vars in 1117 bytes} [Thu Aug 4 21:22:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:38.374361 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816c165-ce7f-4a8f-821a-5742ddd5077e None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:38.527496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816c165-ce7f-4a8f-821a-5742ddd5077e None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:38.565660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad3e87-8c77-407b-bfa5-58c8504ece6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:38.566340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad3e87-8c77-407b-bfa5-58c8504ece6d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:38.566623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad3e87-8c77-407b-bfa5-58c8504ece6d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:38.566901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad3e87-8c77-407b-bfa5-58c8504ece6d None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:38.567695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad3e87-8c77-407b-bfa5-58c8504ece6d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:38.568001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad3e87-8c77-407b-bfa5-58c8504ece6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:38.569433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad3e87-8c77-407b-bfa5-58c8504ece6d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:38.571176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:38.590519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 442/884] 10.222.0.73 () {56 vars in 1149 bytes} [Thu Aug 4 21:22:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:38.596954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a50d5-05de-4c81-aab6-78a8a5ee5111 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:38.675288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:38.738024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a50d5-05de-4c81-aab6-78a8a5ee5111 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:38.786342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ccfdb-9081-4ca0-a898-7b46ee0a8dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:38.787284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ccfdb-9081-4ca0-a898-7b46ee0a8dca None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:38.787878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ccfdb-9081-4ca0-a898-7b46ee0a8dca None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:38.788336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ccfdb-9081-4ca0-a898-7b46ee0a8dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:38.866682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ccfdb-9081-4ca0-a898-7b46ee0a8dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:38.868684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ccfdb-9081-4ca0-a898-7b46ee0a8dca None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:38.939631 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf88a9-ef97-4946-957c-d7cf09574ace None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:38.939995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf88a9-ef97-4946-957c-d7cf09574ace None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:38.940197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf88a9-ef97-4946-957c-d7cf09574ace None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:38.941805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:38.941935 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 443/885] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:38.948089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854edf85-3f58-4a34-9327-d7ba49f13920 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:38.949078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854edf85-3f58-4a34-9327-d7ba49f13920 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:38.949554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854edf85-3f58-4a34-9327-d7ba49f13920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:38.971755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-854edf85-3f58-4a34-9327-d7ba49f13920 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:38.972966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-854edf85-3f58-4a34-9327-d7ba49f13920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:39.037415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 443/886] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:38 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:22:39.038843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ccfdb-9081-4ca0-a898-7b46ee0a8dca None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:39.083231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 444/887] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:39.105758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854edf85-3f58-4a34-9327-d7ba49f13920 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:39.192997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ccfdb-9081-4ca0-a898-7b46ee0a8dca None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:39.221857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854edf85-3f58-4a34-9327-d7ba49f13920 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:39.245186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d4278-0802-4f83-b803-7031ec7fc31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:39.246471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d4278-0802-4f83-b803-7031ec7fc31a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:39.246585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d4278-0802-4f83-b803-7031ec7fc31a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:39.246863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d4278-0802-4f83-b803-7031ec7fc31a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:39.251402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:39.260907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d4278-0802-4f83-b803-7031ec7fc31a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:39.262477 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d4278-0802-4f83-b803-7031ec7fc31a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:39.268870 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3aa18b-7aac-424d-ab26-f3de6f6f81eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:39.271567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa18b-7aac-424d-ab26-f3de6f6f81eb None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:39.271567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa18b-7aac-424d-ab26-f3de6f6f81eb None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:39.271567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa18b-7aac-424d-ab26-f3de6f6f81eb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:39.272154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3aa18b-7aac-424d-ab26-f3de6f6f81eb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:39.274626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3aa18b-7aac-424d-ab26-f3de6f6f81eb None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-3c4d4278-0802-4f83-b803-7031ec7fc31a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:39.279566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:39.280488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:39.281388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:39.281388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:39.281388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:39.281388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:39.286610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 444/888] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:39.288218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d4278-0802-4f83-b803-7031ec7fc31a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-ed3aa18b-7aac-424d-ab26-f3de6f6f81eb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:39.288871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:39.290309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:22:39.291649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:39.298194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 445/889] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:22:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:39.310060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3aa18b-7aac-424d-ab26-f3de6f6f81eb None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:39.310792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:39.417119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d4278-0802-4f83-b803-7031ec7fc31a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:39.457910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3aa18b-7aac-424d-ab26-f3de6f6f81eb None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:39.458458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494df392-9464-404a-9f8b-7310a45c4e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:39.458859 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494df392-9464-404a-9f8b-7310a45c4e6d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:39.459247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494df392-9464-404a-9f8b-7310a45c4e6d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:39.459623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494df392-9464-404a-9f8b-7310a45c4e6d None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:39.460412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494df392-9464-404a-9f8b-7310a45c4e6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:39.461016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494df392-9464-404a-9f8b-7310a45c4e6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:39.462879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494df392-9464-404a-9f8b-7310a45c4e6d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:39.486105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 445/890] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:39.489599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494df392-9464-404a-9f8b-7310a45c4e6d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:39.507181 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9c3f4-c231-407e-b13c-9d4804d3789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:39.507813 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9c3f4-c231-407e-b13c-9d4804d3789a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:39.508099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9c3f4-c231-407e-b13c-9d4804d3789a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:39.508431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9c3f4-c231-407e-b13c-9d4804d3789a None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:39.509428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9c3f4-c231-407e-b13c-9d4804d3789a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:39.509745 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9c3f4-c231-407e-b13c-9d4804d3789a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:39.511682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9c3f4-c231-407e-b13c-9d4804d3789a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:39.522300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 446/891] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:22:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:39.525683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9c3f4-c231-407e-b13c-9d4804d3789a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:39.590603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:39.603608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:39.631312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494df392-9464-404a-9f8b-7310a45c4e6d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:39.657364 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9c3f4-c231-407e-b13c-9d4804d3789a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:39.677786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07252b-393f-4abc-a892-9e5d8037a2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:39.679115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07252b-393f-4abc-a892-9e5d8037a2dc None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:39.679545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07252b-393f-4abc-a892-9e5d8037a2dc None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:39.680267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07252b-393f-4abc-a892-9e5d8037a2dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:39.690996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07252b-393f-4abc-a892-9e5d8037a2dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:39.692460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07252b-393f-4abc-a892-9e5d8037a2dc None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:39.699334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096832de-b5e6-454e-825e-78dda011ca0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:39.700986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096832de-b5e6-454e-825e-78dda011ca0c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:39.701370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096832de-b5e6-454e-825e-78dda011ca0c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:39.701764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096832de-b5e6-454e-825e-78dda011ca0c None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:39.702589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096832de-b5e6-454e-825e-78dda011ca0c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:39.703001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096832de-b5e6-454e-825e-78dda011ca0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:39.704490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096832de-b5e6-454e-825e-78dda011ca0c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:39.715865 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 447/892] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:22:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-7b07252b-393f-4abc-a892-9e5d8037a2dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:39.716013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:39.716847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:39.717801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:39.717801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:39.717801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:39.717801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:39.717801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:39.717801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:39.717801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:39.717801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:39.721132 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096832de-b5e6-454e-825e-78dda011ca0c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:39.721600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 446/893] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:39.723070 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07252b-393f-4abc-a892-9e5d8037a2dc None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:39.830829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096832de-b5e6-454e-825e-78dda011ca0c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:39.831542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07252b-393f-4abc-a892-9e5d8037a2dc None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:39.845731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:39.863902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:39.868389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ff548-38b7-4627-b086-fefbf3c64b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:39.869098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff548-38b7-4627-b086-fefbf3c64b7d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:39.869457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff548-38b7-4627-b086-fefbf3c64b7d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:39.869755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff548-38b7-4627-b086-fefbf3c64b7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:39.870497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c0e77-e17c-4f06-97c2-ff53dc175a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:39.870615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ff548-38b7-4627-b086-fefbf3c64b7d None admin] RBAC: Adding query filter params (interface=public, service_id=c85fa61ee4754c7db741d7d346653518, region_id=RegionOne) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:39.870848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ff548-38b7-4627-b086-fefbf3c64b7d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:39.871700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c0e77-e17c-4f06-97c2-ff53dc175a54 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:39.872191 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c0e77-e17c-4f06-97c2-ff53dc175a54 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:39.872293 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ff548-38b7-4627-b086-fefbf3c64b7d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:39.872745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c0e77-e17c-4f06-97c2-ff53dc175a54 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:39.873689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c0e77-e17c-4f06-97c2-ff53dc175a54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:39.874225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c0e77-e17c-4f06-97c2-ff53dc175a54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:39.875972 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c0e77-e17c-4f06-97c2-ff53dc175a54 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:39.878277 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 448/894] 10.222.0.73 () {56 vars in 1251 bytes} [Thu Aug 4 21:22:39 2022] GET /identity/v3/endpoints?service_id=c85fa61ee4754c7db741d7d346653518&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:39.883426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ff548-38b7-4627-b086-fefbf3c64b7d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:39.900363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 447/895] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:39.906557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c0e77-e17c-4f06-97c2-ff53dc175a54 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:40.011689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c0e77-e17c-4f06-97c2-ff53dc175a54 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:40.012035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ff548-38b7-4627-b086-fefbf3c64b7d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:40.047453 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8690a7-af18-42e4-88e2-dfad11f39356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:40.047954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b90349-74a8-4235-803d-64453166f473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:40.048095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8690a7-af18-42e4-88e2-dfad11f39356 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.048346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8690a7-af18-42e4-88e2-dfad11f39356 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.048573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90349-74a8-4235-803d-64453166f473 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.048732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8690a7-af18-42e4-88e2-dfad11f39356 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.048912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90349-74a8-4235-803d-64453166f473 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.049190 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90349-74a8-4235-803d-64453166f473 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.049377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8690a7-af18-42e4-88e2-dfad11f39356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:40.050099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b90349-74a8-4235-803d-64453166f473 None admin] RBAC: Adding query filter params (role.id=c8d5a540444e458fbccbcad1491b2872, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=6027cfb8625c4d028e7160a85e81bb74) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:40.050418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b90349-74a8-4235-803d-64453166f473 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:40.050768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8690a7-af18-42e4-88e2-dfad11f39356 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:40.051843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b90349-74a8-4235-803d-64453166f473 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:40.070429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 449/896] 10.222.0.73 () {56 vars in 1087 bytes} [Thu Aug 4 21:22:39 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:22:40.070641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 448/897] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:39 2022] GET /identity/v3/role_assignments?user.id=6027cfb8625c4d028e7160a85e81bb74&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=c8d5a540444e458fbccbcad1491b2872 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:40.070738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c826d-193c-495b-8082-9c3ca8346f41 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.070738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c826d-193c-495b-8082-9c3ca8346f41 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.070738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c826d-193c-495b-8082-9c3ca8346f41 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.070738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 450/898] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:40.072262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdb915-ccc3-4989-8879-db06fc0c200f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.072559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdb915-ccc3-4989-8879-db06fc0c200f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.072837 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdb915-ccc3-4989-8879-db06fc0c200f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.096131 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1dbdb915-ccc3-4989-8879-db06fc0c200f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:40.097368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1dbdb915-ccc3-4989-8879-db06fc0c200f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:40.162788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:40.183464 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 449/899] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:40.195661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3020-b66c-4a21-a524-61cd4d5e9f88 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.196301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3020-b66c-4a21-a524-61cd4d5e9f88 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.196778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3020-b66c-4a21-a524-61cd4d5e9f88 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.199282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 451/900] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:40.208163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36aed6-0af1-4cc3-b3bf-bd37cff37b28 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.208163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36aed6-0af1-4cc3-b3bf-bd37cff37b28 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.208163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36aed6-0af1-4cc3-b3bf-bd37cff37b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.213491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af3020-b66c-4a21-a524-61cd4d5e9f88 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:40.228481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-4d36aed6-0af1-4cc3-b3bf-bd37cff37b28 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:40.229809 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4d36aed6-0af1-4cc3-b3bf-bd37cff37b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:40.276587 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:40.309260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 450/901] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:40.333920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36aed6-0af1-4cc3-b3bf-bd37cff37b28 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:40.354048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af3020-b66c-4a21-a524-61cd4d5e9f88 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:40.394261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cdeb73-05da-4e43-86ff-21f4fbc4f7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:40.395249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdeb73-05da-4e43-86ff-21f4fbc4f7d0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.395750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdeb73-05da-4e43-86ff-21f4fbc4f7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.396352 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdeb73-05da-4e43-86ff-21f4fbc4f7d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.397212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cdeb73-05da-4e43-86ff-21f4fbc4f7d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:40.398795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cdeb73-05da-4e43-86ff-21f4fbc4f7d0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-d9cdeb73-05da-4e43-86ff-21f4fbc4f7d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:40.408803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:40.409886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:40.410752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:40.413411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 452/902] 10.222.0.73 () {56 vars in 1126 bytes} [Thu Aug 4 21:22:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:40.419434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cdeb73-05da-4e43-86ff-21f4fbc4f7d0 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:40.466224 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36aed6-0af1-4cc3-b3bf-bd37cff37b28 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:40.488669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:40.513000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da554a07-15a3-4c24-a758-d29148607757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:40.514300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da554a07-15a3-4c24-a758-d29148607757 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.514592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da554a07-15a3-4c24-a758-d29148607757 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.514872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da554a07-15a3-4c24-a758-d29148607757 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.515719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da554a07-15a3-4c24-a758-d29148607757 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:40.517623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da554a07-15a3-4c24-a758-d29148607757 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-da554a07-15a3-4c24-a758-d29148607757 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:40.531686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:40.532695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:22:40.533746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:40.538581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 451/903] 10.222.0.73 () {56 vars in 1132 bytes} [Thu Aug 4 21:22:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:40.539714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da554a07-15a3-4c24-a758-d29148607757 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:40.553380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cdeb73-05da-4e43-86ff-21f4fbc4f7d0 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:40.565048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:40.598811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0fc06-b8ab-4d70-a747-707d3a7911e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:40.600250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0fc06-b8ab-4d70-a747-707d3a7911e7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.600809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0fc06-b8ab-4d70-a747-707d3a7911e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.601355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0fc06-b8ab-4d70-a747-707d3a7911e7 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.602416 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0fc06-b8ab-4d70-a747-707d3a7911e7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:40.603103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0fc06-b8ab-4d70-a747-707d3a7911e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:40.605157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0fc06-b8ab-4d70-a747-707d3a7911e7 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:40.617873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 453/904] 10.222.0.73 () {56 vars in 1122 bytes} [Thu Aug 4 21:22:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:40.619352 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0fc06-b8ab-4d70-a747-707d3a7911e7 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:40.673880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da554a07-15a3-4c24-a758-d29148607757 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:40.720252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050085f-6c1f-41b9-b9e9-5ace87f210e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:40.721157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050085f-6c1f-41b9-b9e9-5ace87f210e6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.721463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050085f-6c1f-41b9-b9e9-5ace87f210e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.721741 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050085f-6c1f-41b9-b9e9-5ace87f210e6 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.722728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050085f-6c1f-41b9-b9e9-5ace87f210e6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:40.723021 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050085f-6c1f-41b9-b9e9-5ace87f210e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:40.725222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050085f-6c1f-41b9-b9e9-5ace87f210e6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:40.733255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0fc06-b8ab-4d70-a747-707d3a7911e7 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:40.737206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 452/905] 10.222.0.73 () {56 vars in 1126 bytes} [Thu Aug 4 21:22:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:40.738401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050085f-6c1f-41b9-b9e9-5ace87f210e6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:40.776316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c77c39-e499-4f10-80f1-1ec5426cf8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:40.777286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77c39-e499-4f10-80f1-1ec5426cf8ec None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.777791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77c39-e499-4f10-80f1-1ec5426cf8ec None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.778286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77c39-e499-4f10-80f1-1ec5426cf8ec None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.779149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c77c39-e499-4f10-80f1-1ec5426cf8ec None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:40.779697 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c77c39-e499-4f10-80f1-1ec5426cf8ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:40.781333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c77c39-e499-4f10-80f1-1ec5426cf8ec None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:40.798729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 454/906] 10.222.0.73 () {56 vars in 1122 bytes} [Thu Aug 4 21:22:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:40.799624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:40.863864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050085f-6c1f-41b9-b9e9-5ace87f210e6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:40.903260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672971c-ae04-46af-ada2-060dfb9a9390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:40.903919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672971c-ae04-46af-ada2-060dfb9a9390 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:40.904183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672971c-ae04-46af-ada2-060dfb9a9390 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:40.904460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672971c-ae04-46af-ada2-060dfb9a9390 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:40.905144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672971c-ae04-46af-ada2-060dfb9a9390 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:40.905449 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672971c-ae04-46af-ada2-060dfb9a9390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:40.906872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672971c-ae04-46af-ada2-060dfb9a9390 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:40.913937 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 453/907] 10.222.0.73 () {56 vars in 1126 bytes} [Thu Aug 4 21:22:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:40.920239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c77c39-e499-4f10-80f1-1ec5426cf8ec None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:40.963289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:41.035093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c77c39-e499-4f10-80f1-1ec5426cf8ec None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:41.071394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609304a-6624-4777-be97-9732425d1003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:41.072332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609304a-6624-4777-be97-9732425d1003 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:41.072809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609304a-6624-4777-be97-9732425d1003 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:41.073255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609304a-6624-4777-be97-9732425d1003 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:41.074211 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609304a-6624-4777-be97-9732425d1003 None admin] RBAC: Adding query filter params (interface=public, service_id=4e0ddf8799584568b9dc0354043dd833, region_id=RegionOne) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:41.074722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609304a-6624-4777-be97-9732425d1003 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:41.076348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609304a-6624-4777-be97-9732425d1003 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:41.082865 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 455/908] 10.222.0.73 () {56 vars in 1251 bytes} [Thu Aug 4 21:22:40 2022] GET /identity/v3/endpoints?service_id=4e0ddf8799584568b9dc0354043dd833&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:41.088626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672971c-ae04-46af-ada2-060dfb9a9390 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:41.197060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672971c-ae04-46af-ada2-060dfb9a9390 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:41.210404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:41.236123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bde33-6031-49b0-85f4-e7783496c540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:41.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bde33-6031-49b0-85f4-e7783496c540 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:41.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bde33-6031-49b0-85f4-e7783496c540 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:41.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bde33-6031-49b0-85f4-e7783496c540 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:41.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bde33-6031-49b0-85f4-e7783496c540 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:41.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bde33-6031-49b0-85f4-e7783496c540 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:41.246524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 454/909] 10.222.0.73 () {56 vars in 1087 bytes} [Thu Aug 4 21:22:41 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:22:42.174130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd85b22-803e-4acc-85b3-2d62442dc99a None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:42.174657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd85b22-803e-4acc-85b3-2d62442dc99a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:42.174735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd85b22-803e-4acc-85b3-2d62442dc99a None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:42.176182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 456/910] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:42.181591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a73de-247a-40c7-9691-e0126757a434 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:42.182427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a73de-247a-40c7-9691-e0126757a434 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:42.182906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a73de-247a-40c7-9691-e0126757a434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:42.201679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c05a73de-247a-40c7-9691-e0126757a434 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:42.203236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c05a73de-247a-40c7-9691-e0126757a434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:42.277410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 455/911] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:42.277748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7a8a9-204c-48d9-b45b-784e80bd77d1 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:42.278057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7a8a9-204c-48d9-b45b-784e80bd77d1 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:42.278340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7a8a9-204c-48d9-b45b-784e80bd77d1 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:42.279946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 457/912] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:22:42.285111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991132-04e7-49fe-b39d-4e6c8d6c1c8d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:42.285419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991132-04e7-49fe-b39d-4e6c8d6c1c8d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:42.285674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991132-04e7-49fe-b39d-4e6c8d6c1c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:42.301081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7a8a9-204c-48d9-b45b-784e80bd77d1 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:42.304624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-32991132-04e7-49fe-b39d-4e6c8d6c1c8d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:42.306816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-32991132-04e7-49fe-b39d-4e6c8d6c1c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:42.399714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 456/913] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:42.418101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:42.424695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7a8a9-204c-48d9-b45b-784e80bd77d1 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:42.428279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32991132-04e7-49fe-b39d-4e6c8d6c1c8d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:42.463997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:42.469755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0ae67-fa0e-4a60-a545-ce168539fc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:42.471443 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0ae67-fa0e-4a60-a545-ce168539fc66 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:42.471973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0ae67-fa0e-4a60-a545-ce168539fc66 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:42.472443 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0ae67-fa0e-4a60-a545-ce168539fc66 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:42.484185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0ae67-fa0e-4a60-a545-ce168539fc66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-f5e0ae67-fa0e-4a60-a545-ce168539fc66 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:42.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:42.491019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:22:42.491910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:42.495137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 458/914] 10.222.0.73 () {56 vars in 1120 bytes} [Thu Aug 4 21:22:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:42.505288 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0ae67-fa0e-4a60-a545-ce168539fc66 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:42.564043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32991132-04e7-49fe-b39d-4e6c8d6c1c8d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:42.620605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd56c5b-5f69-4810-b522-f0db98557c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:42.621499 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56c5b-5f69-4810-b522-f0db98557c63 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:42.621783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56c5b-5f69-4810-b522-f0db98557c63 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:42.622092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56c5b-5f69-4810-b522-f0db98557c63 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:42.623149 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd56c5b-5f69-4810-b522-f0db98557c63 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:42.625016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd56c5b-5f69-4810-b522-f0db98557c63 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:42.633818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0ae67-fa0e-4a60-a545-ce168539fc66 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-9fd56c5b-5f69-4810-b522-f0db98557c63 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:42.638295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:42.639280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:42.640294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 457/915] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:22:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:42.647364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd56c5b-5f69-4810-b522-f0db98557c63 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:42.687667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f76307-bcb9-433d-9f6a-087013e81dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:42.690432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f76307-bcb9-433d-9f6a-087013e81dce None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:42.690707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f76307-bcb9-433d-9f6a-087013e81dce None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:42.690951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f76307-bcb9-433d-9f6a-087013e81dce None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:42.691664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f76307-bcb9-433d-9f6a-087013e81dce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:42.691965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f76307-bcb9-433d-9f6a-087013e81dce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:42.693529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f76307-bcb9-433d-9f6a-087013e81dce None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:42.707956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 459/916] 10.222.0.73 () {56 vars in 1115 bytes} [Thu Aug 4 21:22:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:42.711548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f76307-bcb9-433d-9f6a-087013e81dce None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:42.743178 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:42.778463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:42.785266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd56c5b-5f69-4810-b522-f0db98557c63 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:42.839388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3adec-e97a-4e16-8a22-178573b384d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:42.840768 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3adec-e97a-4e16-8a22-178573b384d1 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:42.840931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3adec-e97a-4e16-8a22-178573b384d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:42.841312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3adec-e97a-4e16-8a22-178573b384d1 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:42.842404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3adec-e97a-4e16-8a22-178573b384d1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:42.842875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3adec-e97a-4e16-8a22-178573b384d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:42.845110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3adec-e97a-4e16-8a22-178573b384d1 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:42.858346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 458/917] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:22:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:42.862197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3adec-e97a-4e16-8a22-178573b384d1 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:42.873900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f76307-bcb9-433d-9f6a-087013e81dce None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:42.924153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752aeb9-ac5e-4d9c-87f7-b856a75e259d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:42.925200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752aeb9-ac5e-4d9c-87f7-b856a75e259d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:42.925696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752aeb9-ac5e-4d9c-87f7-b856a75e259d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:42.926284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752aeb9-ac5e-4d9c-87f7-b856a75e259d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:42.940979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752aeb9-ac5e-4d9c-87f7-b856a75e259d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:42.943061 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752aeb9-ac5e-4d9c-87f7-b856a75e259d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-c752aeb9-ac5e-4d9c-87f7-b856a75e259d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:42.963866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:42.964834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:42.965762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:42.965762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:42.965762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:42.965762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:42.970143 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 460/918] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:42.976287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752aeb9-ac5e-4d9c-87f7-b856a75e259d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:43.024682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3adec-e97a-4e16-8a22-178573b384d1 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:43.066623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095022a-e46a-40c4-8daf-1245c3b3808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:43.067309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095022a-e46a-40c4-8daf-1245c3b3808a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:43.067594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095022a-e46a-40c4-8daf-1245c3b3808a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:43.067941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095022a-e46a-40c4-8daf-1245c3b3808a None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:43.068635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095022a-e46a-40c4-8daf-1245c3b3808a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:43.069007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095022a-e46a-40c4-8daf-1245c3b3808a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:43.070344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:43.070716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095022a-e46a-40c4-8daf-1245c3b3808a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:43.080907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 459/919] 10.222.0.73 () {56 vars in 1114 bytes} [Thu Aug 4 21:22:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:43.086369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68f2d8-5687-4246-bedc-4c059ef0843d None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:43.086369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68f2d8-5687-4246-bedc-4c059ef0843d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:43.086369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68f2d8-5687-4246-bedc-4c059ef0843d None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:43.086369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 460/920] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:43.086369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68f2d8-5687-4246-bedc-4c059ef0843d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:43.109692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752aeb9-ac5e-4d9c-87f7-b856a75e259d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:43.125372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:43.152784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c56f4-6f33-4ee6-af29-64342f8cb3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:43.153808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c56f4-6f33-4ee6-af29-64342f8cb3e9 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:43.154142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c56f4-6f33-4ee6-af29-64342f8cb3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:43.154427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c56f4-6f33-4ee6-af29-64342f8cb3e9 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:43.155127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c56f4-6f33-4ee6-af29-64342f8cb3e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:43.155462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c56f4-6f33-4ee6-af29-64342f8cb3e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:43.157376 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c56f4-6f33-4ee6-af29-64342f8cb3e9 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:43.177007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 461/921] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:43.177007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b732d-b67e-4c85-8243-cdc87cd8682f None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:43.177007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b732d-b67e-4c85-8243-cdc87cd8682f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:43.177007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b732d-b67e-4c85-8243-cdc87cd8682f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:43.199100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-855b732d-b67e-4c85-8243-cdc87cd8682f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:43.200268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-855b732d-b67e-4c85-8243-cdc87cd8682f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:43.221462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68f2d8-5687-4246-bedc-4c059ef0843d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:43.264309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a75dc-f9dd-4cdb-acda-a01bfcc45d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:43.265506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a75dc-f9dd-4cdb-acda-a01bfcc45d68 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:43.265876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a75dc-f9dd-4cdb-acda-a01bfcc45d68 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:43.266233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a75dc-f9dd-4cdb-acda-a01bfcc45d68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:43.266968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a75dc-f9dd-4cdb-acda-a01bfcc45d68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:43.268835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a75dc-f9dd-4cdb-acda-a01bfcc45d68 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:43.276796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 462/922] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:43.277647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b732d-b67e-4c85-8243-cdc87cd8682f None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:43.350744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:43.369542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 461/923] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:22:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:22:43.370956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a75dc-f9dd-4cdb-acda-a01bfcc45d68 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:43.406658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b732d-b67e-4c85-8243-cdc87cd8682f None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:43.468830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912bd3b-a57b-4cab-bd35-55db1a5b8631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:43.469757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912bd3b-a57b-4cab-bd35-55db1a5b8631 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:43.470274 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912bd3b-a57b-4cab-bd35-55db1a5b8631 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:43.471590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912bd3b-a57b-4cab-bd35-55db1a5b8631 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:43.485801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912bd3b-a57b-4cab-bd35-55db1a5b8631 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:43.488646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912bd3b-a57b-4cab-bd35-55db1a5b8631 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-3912bd3b-a57b-4cab-bd35-55db1a5b8631 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:43.511354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:43.512456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:43.513559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:43.513559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:43.513559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:43.513559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:43.513559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:43.513559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:43.513559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:43.513559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:43.513559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:43.513559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:43.513559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 463/924] 10.222.0.73 () {56 vars in 1117 bytes} [Thu Aug 4 21:22:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:43.515552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8353967-df61-4040-81be-f1c29dfcd468 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:43.515919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:43.516044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8353967-df61-4040-81be-f1c29dfcd468 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:43.516434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8353967-df61-4040-81be-f1c29dfcd468 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:43.521426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 464/925] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:43.521764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8353967-df61-4040-81be-f1c29dfcd468 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:43.551409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a75dc-f9dd-4cdb-acda-a01bfcc45d68 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:43.595710 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76336-7061-4798-8771-41a91db13ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:43.596442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76336-7061-4798-8771-41a91db13ab5 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:43.596798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76336-7061-4798-8771-41a91db13ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:43.597044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76336-7061-4798-8771-41a91db13ab5 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:43.597750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc76336-7061-4798-8771-41a91db13ab5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:43.599241 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc76336-7061-4798-8771-41a91db13ab5 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:43.656407 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8353967-df61-4040-81be-f1c29dfcd468 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:43.679257 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:43.686049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 462/926] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:22:43 2022] POST /identity/v3/services => generated 241 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:22:43.686049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275bb4e-659a-44cd-85dd-05ffe285e275 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:43.686049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275bb4e-659a-44cd-85dd-05ffe285e275 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:43.686049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275bb4e-659a-44cd-85dd-05ffe285e275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:43.707452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1275bb4e-659a-44cd-85dd-05ffe285e275 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:43.709313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1275bb4e-659a-44cd-85dd-05ffe285e275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:43.726294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351d360-2db1-4402-a1cf-7a13d73a98d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:43.726852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351d360-2db1-4402-a1cf-7a13d73a98d4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:43.727905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351d360-2db1-4402-a1cf-7a13d73a98d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:43.728490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351d360-2db1-4402-a1cf-7a13d73a98d4 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:43.729907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351d360-2db1-4402-a1cf-7a13d73a98d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:43.730394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351d360-2db1-4402-a1cf-7a13d73a98d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:43.732967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351d360-2db1-4402-a1cf-7a13d73a98d4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:43.764046 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 465/927] 10.222.0.73 () {56 vars in 1149 bytes} [Thu Aug 4 21:22:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:43.770279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351d360-2db1-4402-a1cf-7a13d73a98d4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:43.815266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 463/928] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:43.836573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275bb4e-659a-44cd-85dd-05ffe285e275 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:43.937350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351d360-2db1-4402-a1cf-7a13d73a98d4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:43.954419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:43.978603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275bb4e-659a-44cd-85dd-05ffe285e275 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:43.980899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874f691-b0c4-407e-ad15-893313483e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:43.981650 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874f691-b0c4-407e-ad15-893313483e5a None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:43.982033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874f691-b0c4-407e-ad15-893313483e5a None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:43.982400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874f691-b0c4-407e-ad15-893313483e5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:43.995605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874f691-b0c4-407e-ad15-893313483e5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:43.997257 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874f691-b0c4-407e-ad15-893313483e5a None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-5874f691-b0c4-407e-ad15-893313483e5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:44.015106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:44.016141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:44.017057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:44.017057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:44.017057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:44.017057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:44.017057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 466/929] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:44.022626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874f691-b0c4-407e-ad15-893313483e5a None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:44.025205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68140917-b970-479d-9574-826e72cdfabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:44.026325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140917-b970-479d-9574-826e72cdfabd None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:44.027720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140917-b970-479d-9574-826e72cdfabd None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:44.028076 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140917-b970-479d-9574-826e72cdfabd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:44.028838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68140917-b970-479d-9574-826e72cdfabd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:44.030454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68140917-b970-479d-9574-826e72cdfabd None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-68140917-b970-479d-9574-826e72cdfabd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:44.044585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:44.046595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:44.048818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 464/930] 10.222.0.73 () {56 vars in 1132 bytes} [Thu Aug 4 21:22:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:44.054057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68140917-b970-479d-9574-826e72cdfabd None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:44.061010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:44.161914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874f691-b0c4-407e-ad15-893313483e5a None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:44.200984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68140917-b970-479d-9574-826e72cdfabd None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:44.204648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44288540-8184-4c92-b84f-7b808d2aa148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:44.204648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44288540-8184-4c92-b84f-7b808d2aa148 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:44.205807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44288540-8184-4c92-b84f-7b808d2aa148 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:44.205882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44288540-8184-4c92-b84f-7b808d2aa148 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:44.206702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44288540-8184-4c92-b84f-7b808d2aa148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:44.207113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44288540-8184-4c92-b84f-7b808d2aa148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:44.208864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44288540-8184-4c92-b84f-7b808d2aa148 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:44.231112 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 467/931] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:44.232300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44288540-8184-4c92-b84f-7b808d2aa148 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:44.250487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75de935-e0d8-49af-af80-db26a8e22d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:44.252803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75de935-e0d8-49af-af80-db26a8e22d17 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:44.253659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75de935-e0d8-49af-af80-db26a8e22d17 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:44.254254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75de935-e0d8-49af-af80-db26a8e22d17 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:44.255139 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75de935-e0d8-49af-af80-db26a8e22d17 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:44.255923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75de935-e0d8-49af-af80-db26a8e22d17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:44.258438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75de935-e0d8-49af-af80-db26a8e22d17 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:44.270623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:44.271043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 465/932] 10.222.0.73 () {56 vars in 1126 bytes} [Thu Aug 4 21:22:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:44.272248 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75de935-e0d8-49af-af80-db26a8e22d17 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:44.340780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:44.358989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44288540-8184-4c92-b84f-7b808d2aa148 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:44.390475 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75de935-e0d8-49af-af80-db26a8e22d17 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:44.398323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9953f-c5dd-495f-bbd8-caa8884fc1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:44.399144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9953f-c5dd-495f-bbd8-caa8884fc1c6 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:44.399500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9953f-c5dd-495f-bbd8-caa8884fc1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:44.399853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9953f-c5dd-495f-bbd8-caa8884fc1c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:44.408679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca9953f-c5dd-495f-bbd8-caa8884fc1c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:44.411269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca9953f-c5dd-495f-bbd8-caa8884fc1c6 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:44.432985 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195db65-19c6-4936-9388-821138edb6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:44.433953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195db65-19c6-4936-9388-821138edb6fc None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:44.434453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195db65-19c6-4936-9388-821138edb6fc None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:44.434907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195db65-19c6-4936-9388-821138edb6fc None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-6ca9953f-c5dd-495f-bbd8-caa8884fc1c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:44.435166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:44.436063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:44.437066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:44.437066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:44.437066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:44.437066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:44.437066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:44.437066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:44.437066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:44.437066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:44.438528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195db65-19c6-4936-9388-821138edb6fc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:44.439062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195db65-19c6-4936-9388-821138edb6fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:44.441186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195db65-19c6-4936-9388-821138edb6fc None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:44.444153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 468/933] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:44.447680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 466/934] 10.222.0.73 () {56 vars in 1126 bytes} [Thu Aug 4 21:22:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:44.452403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9953f-c5dd-495f-bbd8-caa8884fc1c6 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:44.453692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195db65-19c6-4936-9388-821138edb6fc None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:44.527684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:44.582878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195db65-19c6-4936-9388-821138edb6fc None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:44.587244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9953f-c5dd-495f-bbd8-caa8884fc1c6 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:44.625114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574bc8-effd-473a-9457-4f9e95b009ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:44.625770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5f94f-8ab5-40d8-ab9a-710676bcf3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:44.625931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574bc8-effd-473a-9457-4f9e95b009ac None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:44.626230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574bc8-effd-473a-9457-4f9e95b009ac None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:44.626583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574bc8-effd-473a-9457-4f9e95b009ac None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:44.627667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5f94f-8ab5-40d8-ab9a-710676bcf3c4 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:44.628128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5f94f-8ab5-40d8-ab9a-710676bcf3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:44.628269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50574bc8-effd-473a-9457-4f9e95b009ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:44.628641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50574bc8-effd-473a-9457-4f9e95b009ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:44.628977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5f94f-8ab5-40d8-ab9a-710676bcf3c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:44.629866 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5f94f-8ab5-40d8-ab9a-710676bcf3c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:44.630344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50574bc8-effd-473a-9457-4f9e95b009ac None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:44.631641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5f94f-8ab5-40d8-ab9a-710676bcf3c4 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:44.646086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 469/935] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:44.651696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574bc8-effd-473a-9457-4f9e95b009ac None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:44.656927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:44.760503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 467/936] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:22:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:22:44.774812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574bc8-effd-473a-9457-4f9e95b009ac None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:44.831160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8970c6e-cc2b-4e13-94db-37f7c3b73a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:44.832294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8970c6e-cc2b-4e13-94db-37f7c3b73a29 None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:44.832818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8970c6e-cc2b-4e13-94db-37f7c3b73a29 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:44.833148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8970c6e-cc2b-4e13-94db-37f7c3b73a29 None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/6027cfb8625c4d028e7160a85e81bb74/roles/c8d5a540444e458fbccbcad1491b2872` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:44.835673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8970c6e-cc2b-4e13-94db-37f7c3b73a29 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=6027cfb8625c4d028e7160a85e81bb74, role_id=c8d5a540444e458fbccbcad1491b2872)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:44.837221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8970c6e-cc2b-4e13-94db-37f7c3b73a29 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:44.913703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 470/937] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:22:44 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/6027cfb8625c4d028e7160a85e81bb74/roles/c8d5a540444e458fbccbcad1491b2872 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:22:45.616448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675e94c-a4ab-4c5b-b7e8-fccbd3c93f87 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:45.616962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675e94c-a4ab-4c5b-b7e8-fccbd3c93f87 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:45.617068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675e94c-a4ab-4c5b-b7e8-fccbd3c93f87 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:45.618687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 468/938] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:45.624034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526925-7958-4d38-b150-d941496a8560 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:45.624455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526925-7958-4d38-b150-d941496a8560 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:45.624644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526925-7958-4d38-b150-d941496a8560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:45.647362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-dd526925-7958-4d38-b150-d941496a8560 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:45.648586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-dd526925-7958-4d38-b150-d941496a8560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:45.709218 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:45.721078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 471/939] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:45.745105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675e94c-a4ab-4c5b-b7e8-fccbd3c93f87 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:45.852744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675e94c-a4ab-4c5b-b7e8-fccbd3c93f87 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:45.871609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:45.889227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac892d-acfc-4a97-a875-dbaaae02fef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:45.889850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac892d-acfc-4a97-a875-dbaaae02fef7 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:45.890302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac892d-acfc-4a97-a875-dbaaae02fef7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:45.890588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac892d-acfc-4a97-a875-dbaaae02fef7 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:45.891284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac892d-acfc-4a97-a875-dbaaae02fef7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:45.893405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac892d-acfc-4a97-a875-dbaaae02fef7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:45.896247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: WARNING keystone.server.flask.common [None req-bfac892d-acfc-4a97-a875-dbaaae02fef7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 21:22:45.905458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e10e49-e104-4b36-9bfd-d6bad9370525 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:45.905750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e10e49-e104-4b36-9bfd-d6bad9370525 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:45.906048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e10e49-e104-4b36-9bfd-d6bad9370525 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:45.907680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 472/940] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:45.915056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304b87d-8548-439c-849f-0efc31260878 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:45.915345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304b87d-8548-439c-849f-0efc31260878 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:45.915622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304b87d-8548-439c-849f-0efc31260878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:45.935595 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-f304b87d-8548-439c-849f-0efc31260878 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:45.936779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f304b87d-8548-439c-849f-0efc31260878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:45.986940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.sql.core [None req-bfac892d-acfc-4a97-a875-dbaaae02fef7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 21:22:45.986940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 21:22:45.986940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [parameters: {'id': '08812940c49d4871bc5cb84b788531c3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 21:22:45.986940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91913) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-bfac892d-acfc-4a97-a875-dbaaae02fef7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 08812940c49d4871bc5cb84b788531c3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 08812940c49d4871bc5cb84b788531c3. Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:22:46.012342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application result.read() Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 04 21:22:46.013309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application yield resource Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise self.value Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application session.commit() Aug 04 21:22:46.014351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self.session.flush() Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application compat.raise_( Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 21:22:46.015284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 21:22:46.016194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application result.read() Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:22:46.016998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [parameters: {'id': '08812940c49d4871bc5cb84b788531c3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:46.018176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:46.021722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:46.024559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:46.024559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 04 21:22:46.024559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 04 21:22:46.024559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:46.024559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:46.024559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 04 21:22:46.024559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 04 21:22:46.024559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 08812940c49d4871bc5cb84b788531c3. Aug 04 21:22:46.024559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:46.024559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 469/941] 10.222.0.73 () {60 vars in 1140 bytes} [Thu Aug 4 21:22:45 2022] POST /identity/v3/projects => generated 262 bytes in 276 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:46.025889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac892d-acfc-4a97-a875-dbaaae02fef7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:46.034147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 473/942] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:46.051467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304b87d-8548-439c-849f-0efc31260878 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:46.151144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac892d-acfc-4a97-a875-dbaaae02fef7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:46.176131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304b87d-8548-439c-849f-0efc31260878 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:46.199886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:46.200068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4cc46-7d67-4c25-8ede-4835440005c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:46.201349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4cc46-7d67-4c25-8ede-4835440005c2 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:46.201637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4cc46-7d67-4c25-8ede-4835440005c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:46.201918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4cc46-7d67-4c25-8ede-4835440005c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:46.212192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4cc46-7d67-4c25-8ede-4835440005c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:46.213842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4cc46-7d67-4c25-8ede-4835440005c2 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:46.218807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b63d7-81cc-4594-9627-5529b82d8987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:46.218807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b63d7-81cc-4594-9627-5529b82d8987 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:46.218807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b63d7-81cc-4594-9627-5529b82d8987 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:46.218807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b63d7-81cc-4594-9627-5529b82d8987 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:46.219267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b63d7-81cc-4594-9627-5529b82d8987 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:46.220254 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b63d7-81cc-4594-9627-5529b82d8987 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-e7a4cc46-7d67-4c25-8ede-4835440005c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:46.226973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:46.227878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:46.228792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:46.228792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:46.228792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:46.228792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:46.228792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:46.228792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:46.228792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:46.228792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:46.233014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 470/943] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-d19b63d7-81cc-4594-9627-5529b82d8987 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:46.233804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:46.234854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:46.235732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:46.237031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4cc46-7d67-4c25-8ede-4835440005c2 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:46.239396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 474/944] 10.222.0.73 () {56 vars in 1126 bytes} [Thu Aug 4 21:22:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:46.246609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b63d7-81cc-4594-9627-5529b82d8987 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:46.286858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:46.357221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4cc46-7d67-4c25-8ede-4835440005c2 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:46.383841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b63d7-81cc-4594-9627-5529b82d8987 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:46.400500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17d815-5339-478a-9d4c-eadc26490cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:46.401529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17d815-5339-478a-9d4c-eadc26490cea None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:46.402088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17d815-5339-478a-9d4c-eadc26490cea None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:46.402586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17d815-5339-478a-9d4c-eadc26490cea None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:46.403514 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17d815-5339-478a-9d4c-eadc26490cea None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:46.404033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17d815-5339-478a-9d4c-eadc26490cea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:46.405674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17d815-5339-478a-9d4c-eadc26490cea None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:46.426265 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48bcc9-6759-46b6-b99e-b46a68c45da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:46.426890 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48bcc9-6759-46b6-b99e-b46a68c45da4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:46.427176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48bcc9-6759-46b6-b99e-b46a68c45da4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:46.427456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48bcc9-6759-46b6-b99e-b46a68c45da4 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:46.428163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48bcc9-6759-46b6-b99e-b46a68c45da4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:46.428473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48bcc9-6759-46b6-b99e-b46a68c45da4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:46.429884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48bcc9-6759-46b6-b99e-b46a68c45da4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:46.430347 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 471/945] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:46 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:46.440422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 475/946] 10.222.0.73 () {56 vars in 1122 bytes} [Thu Aug 4 21:22:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:46.447133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17d815-5339-478a-9d4c-eadc26490cea None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:46.558130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:46.565628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17d815-5339-478a-9d4c-eadc26490cea None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:46.604903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc7e39-993c-476f-9954-d4a23f7547cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:46.605575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc7e39-993c-476f-9954-d4a23f7547cc None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:46.605863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc7e39-993c-476f-9954-d4a23f7547cc None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:46.606182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc7e39-993c-476f-9954-d4a23f7547cc None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:46.606870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc7e39-993c-476f-9954-d4a23f7547cc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:46.607173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc7e39-993c-476f-9954-d4a23f7547cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:46.608535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc7e39-993c-476f-9954-d4a23f7547cc None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:46.614939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 472/947] 10.222.0.73 () {56 vars in 1122 bytes} [Thu Aug 4 21:22:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:46.620773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48bcc9-6759-46b6-b99e-b46a68c45da4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:46.684225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:46.740177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48bcc9-6759-46b6-b99e-b46a68c45da4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:46.775206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf8ca3-042c-4e13-8fb5-baaaed47ab22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:46.776558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf8ca3-042c-4e13-8fb5-baaaed47ab22 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:46.776846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf8ca3-042c-4e13-8fb5-baaaed47ab22 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:46.777116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf8ca3-042c-4e13-8fb5-baaaed47ab22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:46.778463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf8ca3-042c-4e13-8fb5-baaaed47ab22 None admin] RBAC: Adding query filter params (interface=public, service_id=28225dfec951437dbb5852b777ce861f, region_id=RegionOne) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:46.778780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf8ca3-042c-4e13-8fb5-baaaed47ab22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:46.780220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf8ca3-042c-4e13-8fb5-baaaed47ab22 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:46.786712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 476/948] 10.222.0.73 () {56 vars in 1251 bytes} [Thu Aug 4 21:22:46 2022] GET /identity/v3/endpoints?service_id=28225dfec951437dbb5852b777ce861f&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:46.792418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc7e39-993c-476f-9954-d4a23f7547cc None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:46.900342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc7e39-993c-476f-9954-d4a23f7547cc None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:46.918030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05d7f-2825-4843-9d43-0347afe8f7c2 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:46.922965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05d7f-2825-4843-9d43-0347afe8f7c2 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:46.922965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05d7f-2825-4843-9d43-0347afe8f7c2 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:46.922965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 477/949] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:46.927120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a47cf-4cfa-46d8-8b2f-fed20cd8b624 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:46.927402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a47cf-4cfa-46d8-8b2f-fed20cd8b624 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:46.927673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a47cf-4cfa-46d8-8b2f-fed20cd8b624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:46.946151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392b29f-783b-4f2f-9231-41c64d4438b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:46.947345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392b29f-783b-4f2f-9231-41c64d4438b7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:46.947845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392b29f-783b-4f2f-9231-41c64d4438b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:46.948150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-eb3a47cf-4cfa-46d8-8b2f-fed20cd8b624 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:46.948591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392b29f-783b-4f2f-9231-41c64d4438b7 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:46.949475 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392b29f-783b-4f2f-9231-41c64d4438b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:46.949600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-eb3a47cf-4cfa-46d8-8b2f-fed20cd8b624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:46.951366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392b29f-783b-4f2f-9231-41c64d4438b7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:46.958415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 473/950] 10.222.0.73 () {56 vars in 1087 bytes} [Thu Aug 4 21:22:46 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:22:47.038485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 478/951] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:47.061547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392b29f-783b-4f2f-9231-41c64d4438b7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:47.092518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:47.110622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ba991-dec3-4d57-8e7d-70a115f612d5 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:47.111680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ba991-dec3-4d57-8e7d-70a115f612d5 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:47.111991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ba991-dec3-4d57-8e7d-70a115f612d5 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:47.114109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 479/952] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:47.119825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2601c-d461-4d86-afe4-a1b8739d0778 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:47.120117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2601c-d461-4d86-afe4-a1b8739d0778 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:47.120400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2601c-d461-4d86-afe4-a1b8739d0778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:47.139493 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-8da2601c-d461-4d86-afe4-a1b8739d0778 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:47.140687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8da2601c-d461-4d86-afe4-a1b8739d0778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:47.181648 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392b29f-783b-4f2f-9231-41c64d4438b7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:47.182342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:47.222594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 480/953] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:47.228121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92b223-d043-47e3-b191-2937778bd502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:47.229091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92b223-d043-47e3-b191-2937778bd502 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:47.229589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92b223-d043-47e3-b191-2937778bd502 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:47.230107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92b223-d043-47e3-b191-2937778bd502 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:47.230993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92b223-d043-47e3-b191-2937778bd502 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:47.232623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92b223-d043-47e3-b191-2937778bd502 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-ab92b223-d043-47e3-b191-2937778bd502 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:47.243718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:47.244828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:22:47.245767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:47.247525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2601c-d461-4d86-afe4-a1b8739d0778 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:47.248916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 474/954] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:47.255052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92b223-d043-47e3-b191-2937778bd502 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:47.352425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:47.356253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2601c-d461-4d86-afe4-a1b8739d0778 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:47.367013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92b223-d043-47e3-b191-2937778bd502 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:47.403166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb85e4-e40b-465c-bc46-ae165cf9d385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:47.403805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb85e4-e40b-465c-bc46-ae165cf9d385 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:47.404088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb85e4-e40b-465c-bc46-ae165cf9d385 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:47.404367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb85e4-e40b-465c-bc46-ae165cf9d385 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:47.416623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb85e4-e40b-465c-bc46-ae165cf9d385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:47.419230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a50663-e99e-4eb5-93cb-296a95142cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:47.420009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a50663-e99e-4eb5-93cb-296a95142cb4 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:47.420451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a50663-e99e-4eb5-93cb-296a95142cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:47.420877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a50663-e99e-4eb5-93cb-296a95142cb4 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-9edb85e4-e40b-465c-bc46-ae165cf9d385 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:47.421649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:47.422805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:22:47.424301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:47.425218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a50663-e99e-4eb5-93cb-296a95142cb4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:47.425687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a50663-e99e-4eb5-93cb-296a95142cb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:47.427287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a50663-e99e-4eb5-93cb-296a95142cb4 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:47.428379 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 481/955] 10.222.0.73 () {56 vars in 1120 bytes} [Thu Aug 4 21:22:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:47.430121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb85e4-e40b-465c-bc46-ae165cf9d385 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:47.433469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 475/956] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:47.442080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a50663-e99e-4eb5-93cb-296a95142cb4 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:47.473619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:47.560620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb85e4-e40b-465c-bc46-ae165cf9d385 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:47.585122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a50663-e99e-4eb5-93cb-296a95142cb4 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:47.601334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688993a5-515c-4f45-82ee-8a24c1812cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:47.601964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688993a5-515c-4f45-82ee-8a24c1812cf0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:47.602303 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688993a5-515c-4f45-82ee-8a24c1812cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:47.602583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688993a5-515c-4f45-82ee-8a24c1812cf0 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:47.603306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688993a5-515c-4f45-82ee-8a24c1812cf0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:47.603625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688993a5-515c-4f45-82ee-8a24c1812cf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:47.605214 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688993a5-515c-4f45-82ee-8a24c1812cf0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:47.622041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 482/957] 10.222.0.73 () {56 vars in 1115 bytes} [Thu Aug 4 21:22:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:47.631108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688993a5-515c-4f45-82ee-8a24c1812cf0 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:47.632344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ce002-5425-41c3-aaed-e9f9c02bc42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:47.633030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ce002-5425-41c3-aaed-e9f9c02bc42d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:47.633314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ce002-5425-41c3-aaed-e9f9c02bc42d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:47.633647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ce002-5425-41c3-aaed-e9f9c02bc42d None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:47.634383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ce002-5425-41c3-aaed-e9f9c02bc42d None admin] RBAC: Adding query filter params (type=compute) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:47.634695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ce002-5425-41c3-aaed-e9f9c02bc42d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:47.636825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ce002-5425-41c3-aaed-e9f9c02bc42d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:47.650040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 476/958] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:47.762294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:47.774239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688993a5-515c-4f45-82ee-8a24c1812cf0 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:47.812893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188deabe-83be-4760-beb4-b932e81d8ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:47.813950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188deabe-83be-4760-beb4-b932e81d8ee2 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:47.815091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188deabe-83be-4760-beb4-b932e81d8ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:47.815607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188deabe-83be-4760-beb4-b932e81d8ee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:47.827636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188deabe-83be-4760-beb4-b932e81d8ee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:47.829107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188deabe-83be-4760-beb4-b932e81d8ee2 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-188deabe-83be-4760-beb4-b932e81d8ee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:47.845623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:47.847299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:47.848216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:47.848216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:47.848216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:47.848216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:47.848216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 483/959] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:47.854842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ce002-5425-41c3-aaed-e9f9c02bc42d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:47.873316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:47.964080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ce002-5425-41c3-aaed-e9f9c02bc42d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:47.999379 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16caea-8ea7-4889-a4ae-dcef341597d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:47.999999 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16caea-8ea7-4889-a4ae-dcef341597d0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:48.000277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16caea-8ea7-4889-a4ae-dcef341597d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:48.000549 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16caea-8ea7-4889-a4ae-dcef341597d0 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:48.001231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16caea-8ea7-4889-a4ae-dcef341597d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:48.001647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16caea-8ea7-4889-a4ae-dcef341597d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:48.003371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16caea-8ea7-4889-a4ae-dcef341597d0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:48.015304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 477/960] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:48.021705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188deabe-83be-4760-beb4-b932e81d8ee2 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:48.132467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188deabe-83be-4760-beb4-b932e81d8ee2 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:48.173230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeacb2a-a1b2-4296-a6d6-e02f17ad52a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:48.174763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeacb2a-a1b2-4296-a6d6-e02f17ad52a3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:48.175055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeacb2a-a1b2-4296-a6d6-e02f17ad52a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:48.175300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeacb2a-a1b2-4296-a6d6-e02f17ad52a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:48.183429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeacb2a-a1b2-4296-a6d6-e02f17ad52a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:48.184880 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeacb2a-a1b2-4296-a6d6-e02f17ad52a3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-faeacb2a-a1b2-4296-a6d6-e02f17ad52a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:48.197208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:48.198287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:48.199268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:48.199268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:48.199268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:48.199268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:48.199268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:48.199268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:48.199268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:48.199268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:48.199268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 21:22:48.199268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:48.199268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 484/961] 10.222.0.73 () {56 vars in 1117 bytes} [Thu Aug 4 21:22:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:48.207959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16caea-8ea7-4889-a4ae-dcef341597d0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:48.297288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:48.327423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16caea-8ea7-4889-a4ae-dcef341597d0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:48.363185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771ec7f-3d01-4bcc-8007-b8019b57a9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:48.363805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771ec7f-3d01-4bcc-8007-b8019b57a9ae None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:48.364084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771ec7f-3d01-4bcc-8007-b8019b57a9ae None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:48.364357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771ec7f-3d01-4bcc-8007-b8019b57a9ae None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:48.365149 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771ec7f-3d01-4bcc-8007-b8019b57a9ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:48.365461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771ec7f-3d01-4bcc-8007-b8019b57a9ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:48.366902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771ec7f-3d01-4bcc-8007-b8019b57a9ae None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:48.384978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 478/962] 10.222.0.73 () {56 vars in 1149 bytes} [Thu Aug 4 21:22:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:48.390715 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeacb2a-a1b2-4296-a6d6-e02f17ad52a3 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:48.402278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:48.504474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeacb2a-a1b2-4296-a6d6-e02f17ad52a3 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:48.538989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9a78d-de43-4624-ad39-7ebf9c6c3a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:48.539616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9a78d-de43-4624-ad39-7ebf9c6c3a39 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:48.539937 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9a78d-de43-4624-ad39-7ebf9c6c3a39 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:48.540319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9a78d-de43-4624-ad39-7ebf9c6c3a39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:48.608841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9a78d-de43-4624-ad39-7ebf9c6c3a39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:48.610266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9a78d-de43-4624-ad39-7ebf9c6c3a39 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:48.626357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:48.687289 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41a986-0410-416d-8d72-ef758cd8c0a4 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:48.687619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41a986-0410-416d-8d72-ef758cd8c0a4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:48.687835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41a986-0410-416d-8d72-ef758cd8c0a4 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:48.689497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 479/963] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:22:48.696016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc407f-0a4d-4f7c-b16d-b8b560c60094 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:48.696491 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc407f-0a4d-4f7c-b16d-b8b560c60094 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:48.696836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc407f-0a4d-4f7c-b16d-b8b560c60094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:48.714287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 485/964] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:22:48 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:22:48.719862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9a78d-de43-4624-ad39-7ebf9c6c3a39 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:48.723271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-09cc407f-0a4d-4f7c-b16d-b8b560c60094 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:48.725940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-09cc407f-0a4d-4f7c-b16d-b8b560c60094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:48.801577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 480/965] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:48.825958 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc407f-0a4d-4f7c-b16d-b8b560c60094 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:48.838490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9a78d-de43-4624-ad39-7ebf9c6c3a39 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:48.847397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:48.881983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c048e54-d987-43ec-ad21-2e9616a6a4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:48.882774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c048e54-d987-43ec-ad21-2e9616a6a4ea None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:48.883180 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c048e54-d987-43ec-ad21-2e9616a6a4ea None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:48.883870 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c048e54-d987-43ec-ad21-2e9616a6a4ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:48.897496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c048e54-d987-43ec-ad21-2e9616a6a4ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:48.900429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c048e54-d987-43ec-ad21-2e9616a6a4ea None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:48.909740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-9c048e54-d987-43ec-ad21-2e9616a6a4ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:48.918456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:48.929893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:48.930851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:48.930851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:48.930851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:48.930851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:48.930851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 486/966] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:48.930851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c048e54-d987-43ec-ad21-2e9616a6a4ea None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:48.962215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc407f-0a4d-4f7c-b16d-b8b560c60094 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:49.007652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2f1b8-422d-4f7f-9395-cffbb0393e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:49.008352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2f1b8-422d-4f7f-9395-cffbb0393e64 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.008658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2f1b8-422d-4f7f-9395-cffbb0393e64 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.008989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2f1b8-422d-4f7f-9395-cffbb0393e64 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.009662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2f1b8-422d-4f7f-9395-cffbb0393e64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:49.011222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2f1b8-422d-4f7f-9395-cffbb0393e64 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:49.054892 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c048e54-d987-43ec-ad21-2e9616a6a4ea None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:49.088712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.sql.core [None req-86d2f1b8-422d-4f7f-9395-cffbb0393e64 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 21:22:49.088712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 21:22:49.088712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [parameters: {'user_id': '2d72b14af45e4b9ea7be17549787bed4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 21:22:49.088712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91913) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 21:22:49.094454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b61101-9962-4850-9f17-224394bbd4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:49.095081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b61101-9962-4850-9f17-224394bbd4e7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.095887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b61101-9962-4850-9f17-224394bbd4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.096165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b61101-9962-4850-9f17-224394bbd4e7 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.097049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b61101-9962-4850-9f17-224394bbd4e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:49.097357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b61101-9962-4850-9f17-224394bbd4e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:49.098842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b61101-9962-4850-9f17-224394bbd4e7 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-86d2f1b8-422d-4f7f-9395-cffbb0393e64 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application result.read() Aug 04 21:22:49.102551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:22:49.103474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application yield resource Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application next(self.gen) Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise self.value Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application session.commit() Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 21:22:49.104510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self.session.flush() Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application compat.raise_( Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 21:22:49.105440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 21:22:49.107563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application conn.query(q) Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application result.read() Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 21:22:49.109498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [parameters: {'user_id': '2d72b14af45e4b9ea7be17549787bed4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:49.112740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 04 21:22:49.119386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 04 21:22:49.121970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 04 21:22:49.121970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 04 21:22:49.121970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 21:22:49.121970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 21:22:49.121970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 21:22:49.121970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:49.123688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 487/967] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:49.123853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 481/968] 10.222.0.73 () {60 vars in 1131 bytes} [Thu Aug 4 21:22:48 2022] POST /identity/v3/users => generated 160 bytes in 289 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:49.124003 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b61101-9962-4850-9f17-224394bbd4e7 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:49.128474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2f1b8-422d-4f7f-9395-cffbb0393e64 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:49.147200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:49.240910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b61101-9962-4850-9f17-224394bbd4e7 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:49.243159 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2f1b8-422d-4f7f-9395-cffbb0393e64 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:49.270000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:49.282541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bcdc2-41b1-4a5a-a8e6-4db5995c3a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:49.286302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf470c3d-eefe-4392-97dc-e57ed737dfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:49.286302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf470c3d-eefe-4392-97dc-e57ed737dfe0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.286597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bcdc2-41b1-4a5a-a8e6-4db5995c3a8c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.286718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf470c3d-eefe-4392-97dc-e57ed737dfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.286855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bcdc2-41b1-4a5a-a8e6-4db5995c3a8c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.286990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf470c3d-eefe-4392-97dc-e57ed737dfe0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.287122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bcdc2-41b1-4a5a-a8e6-4db5995c3a8c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.294647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf470c3d-eefe-4392-97dc-e57ed737dfe0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:49.296414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf470c3d-eefe-4392-97dc-e57ed737dfe0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:49.301323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bcdc2-41b1-4a5a-a8e6-4db5995c3a8c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:49.304159 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bcdc2-41b1-4a5a-a8e6-4db5995c3a8c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-bf470c3d-eefe-4392-97dc-e57ed737dfe0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:22:49.310775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:49.319142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:49.320119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:22:49.320119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:22:49.320119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:22:49.320119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:22:49.320119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:22:49.320119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:22:49.320119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:22:49.320119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:49.320119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 482/969] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:49.320119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86096d5d-5b85-457d-9871-9e51acd79d33 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.320119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86096d5d-5b85-457d-9871-9e51acd79d33 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.321769 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86096d5d-5b85-457d-9871-9e51acd79d33 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.321769 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 483/970] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:49.321769 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86096d5d-5b85-457d-9871-9e51acd79d33 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-9f3bcdc2-41b1-4a5a-a8e6-4db5995c3a8c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:49.333258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:49.334526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:49.335746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:49.335746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:49.335746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:49.335746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:49.335746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:49.335746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:49.335746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:49.335746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:49.335746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 21:22:49.335746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:49.338267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 488/971] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:22:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:49.340490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49903f-e93f-40b8-8fbe-29e4022e2065 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.342351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49903f-e93f-40b8-8fbe-29e4022e2065 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.343421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49903f-e93f-40b8-8fbe-29e4022e2065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.372389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-3b49903f-e93f-40b8-8fbe-29e4022e2065 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:49.373748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3b49903f-e93f-40b8-8fbe-29e4022e2065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:49.445575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86096d5d-5b85-457d-9871-9e51acd79d33 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:49.465298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 489/972] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:49.465917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49903f-e93f-40b8-8fbe-29e4022e2065 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:49.480619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:49.499573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded609dc-6cf0-4bf8-b8e2-5d7821dfbe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:49.500592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded609dc-6cf0-4bf8-b8e2-5d7821dfbe7b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.501070 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded609dc-6cf0-4bf8-b8e2-5d7821dfbe7b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.501583 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded609dc-6cf0-4bf8-b8e2-5d7821dfbe7b None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.503060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded609dc-6cf0-4bf8-b8e2-5d7821dfbe7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:49.503476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded609dc-6cf0-4bf8-b8e2-5d7821dfbe7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:49.506105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded609dc-6cf0-4bf8-b8e2-5d7821dfbe7b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:49.530215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 484/973] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:22:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:49.530215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded609dc-6cf0-4bf8-b8e2-5d7821dfbe7b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:49.591441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:49.599354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49903f-e93f-40b8-8fbe-29e4022e2065 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:49.648277 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5c9c0-455f-4280-8e39-49c86c485410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:49.648805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c9c0-455f-4280-8e39-49c86c485410 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.649114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c9c0-455f-4280-8e39-49c86c485410 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.649391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c9c0-455f-4280-8e39-49c86c485410 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.650273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5c9c0-455f-4280-8e39-49c86c485410 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:49.650579 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5c9c0-455f-4280-8e39-49c86c485410 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:49.652034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5c9c0-455f-4280-8e39-49c86c485410 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:49.671022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded609dc-6cf0-4bf8-b8e2-5d7821dfbe7b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:49.689823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 490/974] 10.222.0.73 () {56 vars in 1101 bytes} [Thu Aug 4 21:22:49 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:49.690898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5c9c0-455f-4280-8e39-49c86c485410 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:49.739693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94db81-ecf0-42bf-aa25-7a6bf3f1d6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:49.740452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94db81-ecf0-42bf-aa25-7a6bf3f1d6f0 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.740731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94db81-ecf0-42bf-aa25-7a6bf3f1d6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.741011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94db81-ecf0-42bf-aa25-7a6bf3f1d6f0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.741709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94db81-ecf0-42bf-aa25-7a6bf3f1d6f0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:49.743166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94db81-ecf0-42bf-aa25-7a6bf3f1d6f0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-ad94db81-ecf0-42bf-aa25-7a6bf3f1d6f0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:49.758120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:49.759770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:49.760799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 485/975] 10.222.0.73 () {56 vars in 1126 bytes} [Thu Aug 4 21:22:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:49.769008 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94db81-ecf0-42bf-aa25-7a6bf3f1d6f0 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:49.792295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:49.849359 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5c9c0-455f-4280-8e39-49c86c485410 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:49.894581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492dd61-fa79-4941-b02a-cf38859215f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:49.895223 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492dd61-fa79-4941-b02a-cf38859215f1 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.895502 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492dd61-fa79-4941-b02a-cf38859215f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.895774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492dd61-fa79-4941-b02a-cf38859215f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.896569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492dd61-fa79-4941-b02a-cf38859215f1 None admin] RBAC: Adding query filter params (role.id=c8d5a540444e458fbccbcad1491b2872, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=6027cfb8625c4d028e7160a85e81bb74) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:49.896884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492dd61-fa79-4941-b02a-cf38859215f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:49.898356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492dd61-fa79-4941-b02a-cf38859215f1 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:49.903351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94db81-ecf0-42bf-aa25-7a6bf3f1d6f0 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:49.905092 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 491/976] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:22:49 2022] GET /identity/v3/role_assignments?user.id=6027cfb8625c4d028e7160a85e81bb74&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=c8d5a540444e458fbccbcad1491b2872 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:49.906403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc364fdf-fd62-421f-8ab1-4ba4bd4da80e None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.906688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc364fdf-fd62-421f-8ab1-4ba4bd4da80e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.906988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc364fdf-fd62-421f-8ab1-4ba4bd4da80e None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.910330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 492/977] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:49.914981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c156ec5-0a48-4033-a693-1497e68d0828 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.915268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c156ec5-0a48-4033-a693-1497e68d0828 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.915547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c156ec5-0a48-4033-a693-1497e68d0828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.917186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:49.939763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-6c156ec5-0a48-4033-a693-1497e68d0828 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:49.941010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6c156ec5-0a48-4033-a693-1497e68d0828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:49.962948 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a321fd-53e7-4a57-9283-a4fc5feae21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:49.964351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a321fd-53e7-4a57-9283-a4fc5feae21e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:49.965088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a321fd-53e7-4a57-9283-a4fc5feae21e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:49.965506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a321fd-53e7-4a57-9283-a4fc5feae21e None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:49.966555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a321fd-53e7-4a57-9283-a4fc5feae21e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:49.967019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a321fd-53e7-4a57-9283-a4fc5feae21e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:49.969491 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a321fd-53e7-4a57-9283-a4fc5feae21e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:49.980969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 486/978] 10.222.0.73 () {56 vars in 1122 bytes} [Thu Aug 4 21:22:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:49.990638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a321fd-53e7-4a57-9283-a4fc5feae21e None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:50.052333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 493/979] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:50.075622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c156ec5-0a48-4033-a693-1497e68d0828 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:50.132318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a321fd-53e7-4a57-9283-a4fc5feae21e None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:50.153193 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:50.180354 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f4979-2356-4fff-ada6-1baa0058e6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:50.180988 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4979-2356-4fff-ada6-1baa0058e6be None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:50.181272 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4979-2356-4fff-ada6-1baa0058e6be None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:50.181548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4979-2356-4fff-ada6-1baa0058e6be None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:50.182266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f4979-2356-4fff-ada6-1baa0058e6be None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:50.182576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f4979-2356-4fff-ada6-1baa0058e6be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:50.184103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f4979-2356-4fff-ada6-1baa0058e6be None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:50.190314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 487/980] 10.222.0.73 () {56 vars in 1122 bytes} [Thu Aug 4 21:22:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:50.196242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f4979-2356-4fff-ada6-1baa0058e6be None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:50.207372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c156ec5-0a48-4033-a693-1497e68d0828 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:50.237558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:50.251681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b554f-f952-420e-bc35-7468e9fd5bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:50.251681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b554f-f952-420e-bc35-7468e9fd5bd0 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:50.251681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b554f-f952-420e-bc35-7468e9fd5bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:50.251681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b554f-f952-420e-bc35-7468e9fd5bd0 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:50.252272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b554f-f952-420e-bc35-7468e9fd5bd0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:50.253460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b554f-f952-420e-bc35-7468e9fd5bd0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:50.317517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f4979-2356-4fff-ada6-1baa0058e6be None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:50.338042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 494/981] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:22:50 2022] POST /identity/v3/services => generated 242 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:22:50.395235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d487ff1-ee59-4a5a-acce-13c8843ed903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:50.396268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d487ff1-ee59-4a5a-acce-13c8843ed903 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:50.396726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d487ff1-ee59-4a5a-acce-13c8843ed903 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:50.397172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d487ff1-ee59-4a5a-acce-13c8843ed903 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:50.398564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d487ff1-ee59-4a5a-acce-13c8843ed903 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:50.401361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d487ff1-ee59-4a5a-acce-13c8843ed903 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:50.532001 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 488/982] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:22:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:22:52.086757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3417b-a4e8-406c-95f2-1d8093385d66 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:52.087286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3417b-a4e8-406c-95f2-1d8093385d66 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:52.087365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3417b-a4e8-406c-95f2-1d8093385d66 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:52.088807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 495/983] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:22:52.094141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28168fde-2e5e-41bf-9fac-ac6e5559f084 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:52.094608 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28168fde-2e5e-41bf-9fac-ac6e5559f084 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:52.094683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28168fde-2e5e-41bf-9fac-ac6e5559f084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:52.114860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-28168fde-2e5e-41bf-9fac-ac6e5559f084 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:52.116425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-28168fde-2e5e-41bf-9fac-ac6e5559f084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:52.195228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 489/984] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:52.217797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3417b-a4e8-406c-95f2-1d8093385d66 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:52.329409 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3417b-a4e8-406c-95f2-1d8093385d66 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:52.351600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:52.367165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986aa59-9a00-4a3d-9121-797c14d5e5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:52.367818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986aa59-9a00-4a3d-9121-797c14d5e5b2 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:52.368103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986aa59-9a00-4a3d-9121-797c14d5e5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:52.368377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986aa59-9a00-4a3d-9121-797c14d5e5b2 None admin] PATH_INFO: `/v3/services/image` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:52.369039 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986aa59-9a00-4a3d-9121-797c14d5e5b2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:52.370458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986aa59-9a00-4a3d-9121-797c14d5e5b2 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-e986aa59-9a00-4a3d-9121-797c14d5e5b2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:52.380599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:52.381627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:52.382511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 496/985] 10.222.0.73 () {56 vars in 1105 bytes} [Thu Aug 4 21:22:52 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:52.389856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28168fde-2e5e-41bf-9fac-ac6e5559f084 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:52.406849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:52.445984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e9ba4-a3ed-41f6-a121-02a0cbe2d0a6 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:52.446334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e9ba4-a3ed-41f6-a121-02a0cbe2d0a6 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:52.446572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e9ba4-a3ed-41f6-a121-02a0cbe2d0a6 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:52.448063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 497/986] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:22:52.452898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c274f42-a370-481e-bf4e-1b91a1b98795 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:52.453185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c274f42-a370-481e-bf4e-1b91a1b98795 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:52.453463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c274f42-a370-481e-bf4e-1b91a1b98795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:52.472412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-6c274f42-a370-481e-bf4e-1b91a1b98795 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:52.473616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6c274f42-a370-481e-bf4e-1b91a1b98795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:52.504623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28168fde-2e5e-41bf-9fac-ac6e5559f084 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:52.541672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8033d6-72d9-4716-802a-c1829702ff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:52.542326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8033d6-72d9-4716-802a-c1829702ff90 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:52.542608 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8033d6-72d9-4716-802a-c1829702ff90 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:52.542881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8033d6-72d9-4716-802a-c1829702ff90 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:52.543559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8033d6-72d9-4716-802a-c1829702ff90 None admin] RBAC: Adding query filter params (name=image) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:52.543864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8033d6-72d9-4716-802a-c1829702ff90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:52.545232 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8033d6-72d9-4716-802a-c1829702ff90 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:52.550975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 490/987] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:52.557026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8033d6-72d9-4716-802a-c1829702ff90 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:52.570077 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 498/988] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:52.589102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c274f42-a370-481e-bf4e-1b91a1b98795 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:52.650679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:52.672030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8033d6-72d9-4716-802a-c1829702ff90 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:52.694182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c274f42-a370-481e-bf4e-1b91a1b98795 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:52.706400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10931cf2-5693-46e6-a92e-e809548ed98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:52.707012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10931cf2-5693-46e6-a92e-e809548ed98d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:52.707289 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10931cf2-5693-46e6-a92e-e809548ed98d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:52.707560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10931cf2-5693-46e6-a92e-e809548ed98d None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:52.708227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10931cf2-5693-46e6-a92e-e809548ed98d None admin] RBAC: Adding query filter params (type=image) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:52.708531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10931cf2-5693-46e6-a92e-e809548ed98d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:52.708957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:52.709936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10931cf2-5693-46e6-a92e-e809548ed98d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:52.714280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 491/989] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:52 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:52.715439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba250f0-40b8-48c0-919f-3c38c0776ad6 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:52.715722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba250f0-40b8-48c0-919f-3c38c0776ad6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:52.715992 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba250f0-40b8-48c0-919f-3c38c0776ad6 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:52.718048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 492/990] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:52.724528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2fd91-d71d-4811-842d-c75b1c9d0fba None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:52.724528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2fd91-d71d-4811-842d-c75b1c9d0fba None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:52.724750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2fd91-d71d-4811-842d-c75b1c9d0fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:52.731005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a077b-1cff-42f2-9b37-c45c990c0e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:52.731680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a077b-1cff-42f2-9b37-c45c990c0e58 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:52.732036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a077b-1cff-42f2-9b37-c45c990c0e58 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:52.732320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a077b-1cff-42f2-9b37-c45c990c0e58 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:52.732975 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a077b-1cff-42f2-9b37-c45c990c0e58 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:52.734471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a077b-1cff-42f2-9b37-c45c990c0e58 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:52.746053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bdf2fd91-d71d-4811-842d-c75b1c9d0fba None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-922a077b-1cff-42f2-9b37-c45c990c0e58 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:52.746380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:52.747370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:22:52.748216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:52.748967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bdf2fd91-d71d-4811-842d-c75b1c9d0fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:52.749043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 499/991] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:52.755376 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a077b-1cff-42f2-9b37-c45c990c0e58 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:52.840377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 493/992] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:52.881697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2fd91-d71d-4811-842d-c75b1c9d0fba None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:52.894863 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a077b-1cff-42f2-9b37-c45c990c0e58 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:52.951077 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c2735-6f28-4d44-9084-97ef0eb0cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:52.951802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c2735-6f28-4d44-9084-97ef0eb0cc2e None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:52.952135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c2735-6f28-4d44-9084-97ef0eb0cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:52.952505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c2735-6f28-4d44-9084-97ef0eb0cc2e None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:52.953443 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c2735-6f28-4d44-9084-97ef0eb0cc2e None admin] RBAC: Adding query filter params (name=compute) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:52.953891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c2735-6f28-4d44-9084-97ef0eb0cc2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:52.955889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c2735-6f28-4d44-9084-97ef0eb0cc2e None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:52.962351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:52.963100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 500/993] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:52.972325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c2735-6f28-4d44-9084-97ef0eb0cc2e None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:52.999028 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2fd91-d71d-4811-842d-c75b1c9d0fba None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:53.024220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:53.042137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb6e34-5512-4dc5-a4e3-75bc018c5207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:53.042825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb6e34-5512-4dc5-a4e3-75bc018c5207 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:53.043164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb6e34-5512-4dc5-a4e3-75bc018c5207 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:53.043461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb6e34-5512-4dc5-a4e3-75bc018c5207 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:53.044144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb6e34-5512-4dc5-a4e3-75bc018c5207 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:53.045613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb6e34-5512-4dc5-a4e3-75bc018c5207 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-8bdb6e34-5512-4dc5-a4e3-75bc018c5207 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:53.055360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:53.056882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:53.058292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 494/994] 10.222.0.73 () {56 vars in 1126 bytes} [Thu Aug 4 21:22:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:53.063020 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb6e34-5512-4dc5-a4e3-75bc018c5207 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:53.095126 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c2735-6f28-4d44-9084-97ef0eb0cc2e None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:53.132037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46cca1-5010-4482-9fb9-7be9ba046025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:53.132660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46cca1-5010-4482-9fb9-7be9ba046025 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:53.132945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46cca1-5010-4482-9fb9-7be9ba046025 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:53.133245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46cca1-5010-4482-9fb9-7be9ba046025 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:53.133902 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46cca1-5010-4482-9fb9-7be9ba046025 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:53.134235 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46cca1-5010-4482-9fb9-7be9ba046025 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:53.135612 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46cca1-5010-4482-9fb9-7be9ba046025 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:53.141489 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 501/995] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:53.147640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46cca1-5010-4482-9fb9-7be9ba046025 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:53.175452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb6e34-5512-4dc5-a4e3-75bc018c5207 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:53.217602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2dbb8-225a-4d0c-b169-b3ec48486ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:53.218175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2dbb8-225a-4d0c-b169-b3ec48486ddc None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:53.218459 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2dbb8-225a-4d0c-b169-b3ec48486ddc None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:53.218733 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2dbb8-225a-4d0c-b169-b3ec48486ddc None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:53.219415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2dbb8-225a-4d0c-b169-b3ec48486ddc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:53.219718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2dbb8-225a-4d0c-b169-b3ec48486ddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:53.221656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2dbb8-225a-4d0c-b169-b3ec48486ddc None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:53.227173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:53.230636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 495/996] 10.222.0.73 () {56 vars in 1122 bytes} [Thu Aug 4 21:22:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:53.237019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2dbb8-225a-4d0c-b169-b3ec48486ddc None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:53.264917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46cca1-5010-4482-9fb9-7be9ba046025 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:53.274281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:53.305859 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c05cf-0dc3-4ec7-8492-800644f96811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:53.307178 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c05cf-0dc3-4ec7-8492-800644f96811 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:53.307178 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c05cf-0dc3-4ec7-8492-800644f96811 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:53.307505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c05cf-0dc3-4ec7-8492-800644f96811 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:53.308245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c05cf-0dc3-4ec7-8492-800644f96811 None admin] RBAC: Adding query filter params (interface=public, service_id=d9af2141ce8c4b20b6b36e880861f736, region_id=RegionOne) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:53.308960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c05cf-0dc3-4ec7-8492-800644f96811 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:53.311154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c05cf-0dc3-4ec7-8492-800644f96811 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:53.324180 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 502/997] 10.222.0.73 () {56 vars in 1251 bytes} [Thu Aug 4 21:22:53 2022] GET /identity/v3/endpoints?service_id=d9af2141ce8c4b20b6b36e880861f736&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:53.326439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c05cf-0dc3-4ec7-8492-800644f96811 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:53.372227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2dbb8-225a-4d0c-b169-b3ec48486ddc None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:53.410667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3518e5-8691-4982-a692-d09e7ebcf6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:53.411285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3518e5-8691-4982-a692-d09e7ebcf6f9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:53.411576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3518e5-8691-4982-a692-d09e7ebcf6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:53.411839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3518e5-8691-4982-a692-d09e7ebcf6f9 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:53.412513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3518e5-8691-4982-a692-d09e7ebcf6f9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:53.412855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3518e5-8691-4982-a692-d09e7ebcf6f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:53.414508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3518e5-8691-4982-a692-d09e7ebcf6f9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:53.419512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 496/998] 10.222.0.73 () {56 vars in 1122 bytes} [Thu Aug 4 21:22:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:53.426528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3518e5-8691-4982-a692-d09e7ebcf6f9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:53.450500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c05cf-0dc3-4ec7-8492-800644f96811 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:53.489823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d74895-2310-414e-ad4e-df5760226c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:53.490428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d74895-2310-414e-ad4e-df5760226c54 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:53.490709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d74895-2310-414e-ad4e-df5760226c54 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:53.490982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d74895-2310-414e-ad4e-df5760226c54 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:53.491630 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d74895-2310-414e-ad4e-df5760226c54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:53.493057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d74895-2310-414e-ad4e-df5760226c54 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:53.497091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:53.510044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 503/999] 10.222.0.73 () {56 vars in 1087 bytes} [Thu Aug 4 21:22:53 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:22:53.560593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3518e5-8691-4982-a692-d09e7ebcf6f9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:53.599960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6337a2-dcbb-4456-bccb-b408ce70fe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:53.600494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6337a2-dcbb-4456-bccb-b408ce70fe67 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:53.600776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6337a2-dcbb-4456-bccb-b408ce70fe67 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:53.601051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6337a2-dcbb-4456-bccb-b408ce70fe67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:53.601815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6337a2-dcbb-4456-bccb-b408ce70fe67 None admin] RBAC: Adding query filter params (interface=admin, service_id=28225dfec951437dbb5852b777ce861f, region_id=RegionOne) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:53.602242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6337a2-dcbb-4456-bccb-b408ce70fe67 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:53.603626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6337a2-dcbb-4456-bccb-b408ce70fe67 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:53.615193 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 497/1000] 10.222.0.73 () {56 vars in 1249 bytes} [Thu Aug 4 21:22:53 2022] GET /identity/v3/endpoints?service_id=28225dfec951437dbb5852b777ce861f&interface=admin®ion_id=RegionOne => generated 193 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:53.616386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d74895-2310-414e-ad4e-df5760226c54 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:53.655757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:53.724636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d74895-2310-414e-ad4e-df5760226c54 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:53.761736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb8bc4-d404-4bd6-bcb1-33c6aa08add3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:53.762300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb8bc4-d404-4bd6-bcb1-33c6aa08add3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:53.762583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb8bc4-d404-4bd6-bcb1-33c6aa08add3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:53.762858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb8bc4-d404-4bd6-bcb1-33c6aa08add3 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:53.763510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb8bc4-d404-4bd6-bcb1-33c6aa08add3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:53.764884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb8bc4-d404-4bd6-bcb1-33c6aa08add3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:53.778091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 504/1001] 10.222.0.73 () {56 vars in 1087 bytes} [Thu Aug 4 21:22:53 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:22:54.938630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775886a7-06de-42f2-9710-2bbab8749c6e None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:54.939794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775886a7-06de-42f2-9710-2bbab8749c6e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:54.939877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775886a7-06de-42f2-9710-2bbab8749c6e None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:54.941333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 498/1002] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:54.947128 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d0bfe-ddd9-4e5c-920b-89739d757195 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:54.947696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d0bfe-ddd9-4e5c-920b-89739d757195 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:54.947805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d0bfe-ddd9-4e5c-920b-89739d757195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:54.969481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-305d0bfe-ddd9-4e5c-920b-89739d757195 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:54.971262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-305d0bfe-ddd9-4e5c-920b-89739d757195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:55.059316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 505/1003] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:55.087678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775886a7-06de-42f2-9710-2bbab8749c6e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:55.199047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775886a7-06de-42f2-9710-2bbab8749c6e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:55.227284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:55.237634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3e94a-34a8-4777-8188-79c28b813af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:55.238286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3e94a-34a8-4777-8188-79c28b813af7 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:55.240054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3e94a-34a8-4777-8188-79c28b813af7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:55.240054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3e94a-34a8-4777-8188-79c28b813af7 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:55.240054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3e94a-34a8-4777-8188-79c28b813af7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:55.240932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3e94a-34a8-4777-8188-79c28b813af7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:55.382128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 499/1004] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:22:55 2022] POST /identity/v3/services => generated 242 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:22:55.636255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306172bd-b6b5-44bd-8516-86547d55d0cb None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:55.636609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306172bd-b6b5-44bd-8516-86547d55d0cb None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:55.636832 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306172bd-b6b5-44bd-8516-86547d55d0cb None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:55.638292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 506/1005] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:55.644949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815dfd8f-6a70-4384-a4bb-9414af7c414c None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:55.645363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815dfd8f-6a70-4384-a4bb-9414af7c414c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:55.645721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815dfd8f-6a70-4384-a4bb-9414af7c414c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:55.665428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-815dfd8f-6a70-4384-a4bb-9414af7c414c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:55.666667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-815dfd8f-6a70-4384-a4bb-9414af7c414c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:55.749731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 500/1006] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:55 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:55.773450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306172bd-b6b5-44bd-8516-86547d55d0cb None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:55.807341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:55.842162 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbbbeb-ca8e-4a99-8997-e4189ba30b7c None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:55.842448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbbbeb-ca8e-4a99-8997-e4189ba30b7c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:55.842725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbbbeb-ca8e-4a99-8997-e4189ba30b7c None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:55.844077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 501/1007] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:22:55.849767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdd95c-56c1-4a58-be1e-28de2f0f81e7 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:55.850080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdd95c-56c1-4a58-be1e-28de2f0f81e7 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:55.850356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdd95c-56c1-4a58-be1e-28de2f0f81e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:55.870387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-17bdd95c-56c1-4a58-be1e-28de2f0f81e7 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:55.871572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-17bdd95c-56c1-4a58-be1e-28de2f0f81e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:55.899764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306172bd-b6b5-44bd-8516-86547d55d0cb None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:55.948690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df592fa-11d0-4b0e-a402-d2ef0b57f9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:55.950203 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df592fa-11d0-4b0e-a402-d2ef0b57f9b7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:55.950795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df592fa-11d0-4b0e-a402-d2ef0b57f9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:55.951247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df592fa-11d0-4b0e-a402-d2ef0b57f9b7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:55.952032 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df592fa-11d0-4b0e-a402-d2ef0b57f9b7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:55.952704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 502/1008] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:55 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:22:55.954261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df592fa-11d0-4b0e-a402-d2ef0b57f9b7 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-2df592fa-11d0-4b0e-a402-d2ef0b57f9b7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:55.964501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:55.965758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 21:22:55.966847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:55.969342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 507/1009] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:22:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:55.975968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdd95c-56c1-4a58-be1e-28de2f0f81e7 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:55.976276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df592fa-11d0-4b0e-a402-d2ef0b57f9b7 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:56.039849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:56.088307 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:56.099149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df592fa-11d0-4b0e-a402-d2ef0b57f9b7 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:56.103885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdd95c-56c1-4a58-be1e-28de2f0f81e7 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:56.139853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3667b4-00fe-4646-85c4-b1fe34c6f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:56.140926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3667b4-00fe-4646-85c4-b1fe34c6f4bd None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:56.141447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3667b4-00fe-4646-85c4-b1fe34c6f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:56.142548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3667b4-00fe-4646-85c4-b1fe34c6f4bd None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:56.143534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3667b4-00fe-4646-85c4-b1fe34c6f4bd None admin] RBAC: Adding query filter params (name=compute) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:56.144091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3667b4-00fe-4646-85c4-b1fe34c6f4bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:56.144439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46959d-0f4f-4f8f-ab4c-170fd2dd9fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:56.145086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46959d-0f4f-4f8f-ab4c-170fd2dd9fc9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:56.145365 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46959d-0f4f-4f8f-ab4c-170fd2dd9fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:56.145637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46959d-0f4f-4f8f-ab4c-170fd2dd9fc9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:56.146291 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46959d-0f4f-4f8f-ab4c-170fd2dd9fc9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:56.146711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3667b4-00fe-4646-85c4-b1fe34c6f4bd None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:56.147644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46959d-0f4f-4f8f-ab4c-170fd2dd9fc9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-ec46959d-0f4f-4f8f-ab4c-170fd2dd9fc9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:56.157206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:56.159122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 21:22:56.160048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:56.160800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 508/1010] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:56.160883 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 503/1011] 10.222.0.73 () {56 vars in 1126 bytes} [Thu Aug 4 21:22:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:56.163120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3667b4-00fe-4646-85c4-b1fe34c6f4bd None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:56.164917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46959d-0f4f-4f8f-ab4c-170fd2dd9fc9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:56.272415 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3667b4-00fe-4646-85c4-b1fe34c6f4bd None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:56.281281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46959d-0f4f-4f8f-ab4c-170fd2dd9fc9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:56.293976 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:56.317782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3c18e-8343-4633-b585-a365dd1a9cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:56.318436 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c18e-8343-4633-b585-a365dd1a9cd1 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:56.318716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c18e-8343-4633-b585-a365dd1a9cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:56.319053 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c18e-8343-4633-b585-a365dd1a9cd1 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:56.319755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3c18e-8343-4633-b585-a365dd1a9cd1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:56.320050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3c18e-8343-4633-b585-a365dd1a9cd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:56.321420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3c18e-8343-4633-b585-a365dd1a9cd1 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:56.323339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bd17e-b5e1-482b-98a0-e3855e1ade6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:56.324256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bd17e-b5e1-482b-98a0-e3855e1ade6c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:56.324542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bd17e-b5e1-482b-98a0-e3855e1ade6c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:56.324848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bd17e-b5e1-482b-98a0-e3855e1ade6c None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:56.325532 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bd17e-b5e1-482b-98a0-e3855e1ade6c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:56.325842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bd17e-b5e1-482b-98a0-e3855e1ade6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:56.327202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 509/1012] 10.222.0.73 () {56 vars in 1112 bytes} [Thu Aug 4 21:22:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:56.327421 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bd17e-b5e1-482b-98a0-e3855e1ade6c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:56.332684 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3c18e-8343-4633-b585-a365dd1a9cd1 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:56.337559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 504/1013] 10.222.0.73 () {56 vars in 1122 bytes} [Thu Aug 4 21:22:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:56.338752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bd17e-b5e1-482b-98a0-e3855e1ade6c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:56.359150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:56.450975 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3c18e-8343-4633-b585-a365dd1a9cd1 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:56.456518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bd17e-b5e1-482b-98a0-e3855e1ade6c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:56.489406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc56f6b-b2a8-4d25-8754-1f8c473cb443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:56.490060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc56f6b-b2a8-4d25-8754-1f8c473cb443 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:56.490393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc56f6b-b2a8-4d25-8754-1f8c473cb443 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:56.490668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc56f6b-b2a8-4d25-8754-1f8c473cb443 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:56.491407 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc56f6b-b2a8-4d25-8754-1f8c473cb443 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:56.492863 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc56f6b-b2a8-4d25-8754-1f8c473cb443 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:56.496030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457be64-f8e6-484f-a1e4-9ca1c0be13b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:56.496649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457be64-f8e6-484f-a1e4-9ca1c0be13b2 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:56.496932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457be64-f8e6-484f-a1e4-9ca1c0be13b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:56.497204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457be64-f8e6-484f-a1e4-9ca1c0be13b2 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:56.497892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457be64-f8e6-484f-a1e4-9ca1c0be13b2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:56.498225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457be64-f8e6-484f-a1e4-9ca1c0be13b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:56.499587 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457be64-f8e6-484f-a1e4-9ca1c0be13b2 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:56.506160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 505/1014] 10.222.0.73 () {56 vars in 1122 bytes} [Thu Aug 4 21:22:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:56.511991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457be64-f8e6-484f-a1e4-9ca1c0be13b2 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:56.551244 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:56.582507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 510/1015] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:22:56 2022] POST /identity/v3/endpoints => generated 334 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:22:56.626168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457be64-f8e6-484f-a1e4-9ca1c0be13b2 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:56.675392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bb98f-9f19-4fd3-9997-5058ac5066c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:56.676045 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bb98f-9f19-4fd3-9997-5058ac5066c5 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:56.676508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bb98f-9f19-4fd3-9997-5058ac5066c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:56.676968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bb98f-9f19-4fd3-9997-5058ac5066c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:56.678374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bb98f-9f19-4fd3-9997-5058ac5066c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:56.680067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bb98f-9f19-4fd3-9997-5058ac5066c5 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:56.795426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 506/1016] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:22:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:22:57.482292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76ffa6-567c-4b65-98cb-59f60e81b763 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:57.482811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76ffa6-567c-4b65-98cb-59f60e81b763 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:57.483164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76ffa6-567c-4b65-98cb-59f60e81b763 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:57.484535 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 511/1017] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:57.490839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ebbbb-0605-4199-86d5-1af54bf30df8 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:57.491290 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ebbbb-0605-4199-86d5-1af54bf30df8 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:57.491391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ebbbb-0605-4199-86d5-1af54bf30df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:57.510728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-a62ebbbb-0605-4199-86d5-1af54bf30df8 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:57.511950 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a62ebbbb-0605-4199-86d5-1af54bf30df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:57.594652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 507/1018] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:57.618110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76ffa6-567c-4b65-98cb-59f60e81b763 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:57.702236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:57.729063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76ffa6-567c-4b65-98cb-59f60e81b763 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:57.767721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fcffa-b6ad-47e7-9bb5-549715d9d940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:57.768316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fcffa-b6ad-47e7-9bb5-549715d9d940 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:57.768584 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fcffa-b6ad-47e7-9bb5-549715d9d940 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:57.768874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fcffa-b6ad-47e7-9bb5-549715d9d940 None admin] PATH_INFO: `/v3/services/image` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:57.769487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fcffa-b6ad-47e7-9bb5-549715d9d940 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:57.770888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fcffa-b6ad-47e7-9bb5-549715d9d940 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-6e8fcffa-b6ad-47e7-9bb5-549715d9d940 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:57.781821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:57.783960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:22:57.786286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:57.789370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 512/1019] 10.222.0.73 () {56 vars in 1105 bytes} [Thu Aug 4 21:22:57 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:57.792580 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ebbbb-0605-4199-86d5-1af54bf30df8 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:57.813514 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:57.914785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ebbbb-0605-4199-86d5-1af54bf30df8 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:57.960145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2848c-abef-426c-8bad-cb32f279630f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:57.960791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2848c-abef-426c-8bad-cb32f279630f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:57.961081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2848c-abef-426c-8bad-cb32f279630f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:57.961361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2848c-abef-426c-8bad-cb32f279630f None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:57.962149 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2848c-abef-426c-8bad-cb32f279630f None admin] RBAC: Adding query filter params (name=image) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:57.962469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2848c-abef-426c-8bad-cb32f279630f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:57.963958 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2848c-abef-426c-8bad-cb32f279630f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:58.004699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 508/1020] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:58.011549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fcffa-b6ad-47e7-9bb5-549715d9d940 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:58.127509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fcffa-b6ad-47e7-9bb5-549715d9d940 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:58.169792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be11a6f-62ef-4560-a8ee-fbd29a6b8126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:58.170453 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be11a6f-62ef-4560-a8ee-fbd29a6b8126 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:58.170736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be11a6f-62ef-4560-a8ee-fbd29a6b8126 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:58.171101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be11a6f-62ef-4560-a8ee-fbd29a6b8126 None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:58.171898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be11a6f-62ef-4560-a8ee-fbd29a6b8126 None admin] RBAC: Adding query filter params (type=image) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:58.172208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be11a6f-62ef-4560-a8ee-fbd29a6b8126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:58.173976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be11a6f-62ef-4560-a8ee-fbd29a6b8126 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:58.185195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 513/1021] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:58.187194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2848c-abef-426c-8bad-cb32f279630f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:58.302036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:58.324787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2848c-abef-426c-8bad-cb32f279630f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:58.371017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5a870-4fd2-4e0e-9e77-a831a99d93a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:58.371802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5a870-4fd2-4e0e-9e77-a831a99d93a8 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:58.372205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5a870-4fd2-4e0e-9e77-a831a99d93a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:58.372548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5a870-4fd2-4e0e-9e77-a831a99d93a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:58.373552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5a870-4fd2-4e0e-9e77-a831a99d93a8 None admin] RBAC: Adding query filter params (interface=public, service_id=7511938dd9b44e81bb1a7e04c265c7dc, region_id=RegionOne) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:58.373999 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5a870-4fd2-4e0e-9e77-a831a99d93a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:58.375692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5a870-4fd2-4e0e-9e77-a831a99d93a8 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:58.387148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 509/1022] 10.222.0.73 () {56 vars in 1251 bytes} [Thu Aug 4 21:22:58 2022] GET /identity/v3/endpoints?service_id=7511938dd9b44e81bb1a7e04c265c7dc&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:58.395866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be11a6f-62ef-4560-a8ee-fbd29a6b8126 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:58.401444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:58.551620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be11a6f-62ef-4560-a8ee-fbd29a6b8126 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:58.599910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f8e1-5e62-4aed-86af-cff26ebb90fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:58.600910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f8e1-5e62-4aed-86af-cff26ebb90fe None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:58.601423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f8e1-5e62-4aed-86af-cff26ebb90fe None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:58.601899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f8e1-5e62-4aed-86af-cff26ebb90fe None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:58.602932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05f8e1-5e62-4aed-86af-cff26ebb90fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:58.604517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05f8e1-5e62-4aed-86af-cff26ebb90fe None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:58.613455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 514/1023] 10.222.0.73 () {56 vars in 1087 bytes} [Thu Aug 4 21:22:58 2022] GET /identity/v3/services => generated 1957 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:22:58.802117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b06f8-fd44-4f67-ab8e-72c12d6efe71 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:58.802646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b06f8-fd44-4f67-ab8e-72c12d6efe71 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:58.802751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b06f8-fd44-4f67-ab8e-72c12d6efe71 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:58.807668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 510/1024] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:58.815031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603c757-7708-4d2f-9f9a-6c60c66fa4cc None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:58.815561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603c757-7708-4d2f-9f9a-6c60c66fa4cc None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:58.815803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603c757-7708-4d2f-9f9a-6c60c66fa4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:58.837459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c603c757-7708-4d2f-9f9a-6c60c66fa4cc None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:58.838728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c603c757-7708-4d2f-9f9a-6c60c66fa4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:58.912497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:58.919582 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118ad06-9499-44e7-af56-e1c71a7a9f3b None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:58.919783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 515/1025] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:58.919867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118ad06-9499-44e7-af56-e1c71a7a9f3b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:58.920128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118ad06-9499-44e7-af56-e1c71a7a9f3b None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:58.921629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 511/1026] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:22:58.927872 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409b45f-3bf6-4062-8e34-1f659946a4e4 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:58.928157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409b45f-3bf6-4062-8e34-1f659946a4e4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:58.928427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409b45f-3bf6-4062-8e34-1f659946a4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:58.945806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118ad06-9499-44e7-af56-e1c71a7a9f3b None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:58.948517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-2409b45f-3bf6-4062-8e34-1f659946a4e4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:22:58.949699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2409b45f-3bf6-4062-8e34-1f659946a4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:22:59.025748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 516/1027] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:22:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:22:59.049838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409b45f-3bf6-4062-8e34-1f659946a4e4 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:59.063933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118ad06-9499-44e7-af56-e1c71a7a9f3b None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:59.103458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56dd9f-86e4-414d-8c1d-f192d45e521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:59.104471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56dd9f-86e4-414d-8c1d-f192d45e521b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:59.105011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56dd9f-86e4-414d-8c1d-f192d45e521b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:59.105011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56dd9f-86e4-414d-8c1d-f192d45e521b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:59.119286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56dd9f-86e4-414d-8c1d-f192d45e521b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-3a56dd9f-86e4-414d-8c1d-f192d45e521b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:22:59.124609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:59.125578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:22:59.126529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:59.128607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 512/1028] 10.222.0.73 () {56 vars in 1120 bytes} [Thu Aug 4 21:22:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:59.135988 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56dd9f-86e4-414d-8c1d-f192d45e521b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:59.147260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:59.176938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409b45f-3bf6-4062-8e34-1f659946a4e4 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:59.217508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:59.229865 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88dc27-27ee-451a-bdd4-dfb10932a047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:59.230969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88dc27-27ee-451a-bdd4-dfb10932a047 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:59.231386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88dc27-27ee-451a-bdd4-dfb10932a047 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:59.231985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88dc27-27ee-451a-bdd4-dfb10932a047 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:59.233363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88dc27-27ee-451a-bdd4-dfb10932a047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:59.234877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88dc27-27ee-451a-bdd4-dfb10932a047 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:59.237628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 517/1029] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:59.245463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88dc27-27ee-451a-bdd4-dfb10932a047 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:59.249016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56dd9f-86e4-414d-8c1d-f192d45e521b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:59.291248 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bfbe9-ed6b-4e63-b837-346cc09fdab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:59.291876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bfbe9-ed6b-4e63-b837-346cc09fdab2 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:59.292160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bfbe9-ed6b-4e63-b837-346cc09fdab2 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:59.292436 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bfbe9-ed6b-4e63-b837-346cc09fdab2 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:59.293159 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bfbe9-ed6b-4e63-b837-346cc09fdab2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:59.293466 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bfbe9-ed6b-4e63-b837-346cc09fdab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:59.294878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bfbe9-ed6b-4e63-b837-346cc09fdab2 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:59.304918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 513/1030] 10.222.0.73 () {56 vars in 1115 bytes} [Thu Aug 4 21:22:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:22:59.310485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bfbe9-ed6b-4e63-b837-346cc09fdab2 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:59.363481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88dc27-27ee-451a-bdd4-dfb10932a047 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:59.383854 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:59.407488 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e237e-744c-45ac-98e2-2064e10fc379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:59.408134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e237e-744c-45ac-98e2-2064e10fc379 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:59.408419 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e237e-744c-45ac-98e2-2064e10fc379 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:59.408691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e237e-744c-45ac-98e2-2064e10fc379 None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:59.409387 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e237e-744c-45ac-98e2-2064e10fc379 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:59.411050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e237e-744c-45ac-98e2-2064e10fc379 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:59.420926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bfbe9-ed6b-4e63-b837-346cc09fdab2 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:59.456874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d31cb7-de9c-4934-bb2d-af6788a6daff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:59.457531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d31cb7-de9c-4934-bb2d-af6788a6daff None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:59.457811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d31cb7-de9c-4934-bb2d-af6788a6daff None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:59.458115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d31cb7-de9c-4934-bb2d-af6788a6daff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:59.476551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d31cb7-de9c-4934-bb2d-af6788a6daff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:59.478084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d31cb7-de9c-4934-bb2d-af6788a6daff None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-82d31cb7-de9c-4934-bb2d-af6788a6daff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:22:59.493775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:59.494886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:59.495801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:22:59.495801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:22:59.495801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:22:59.495801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:22:59.495801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 514/1031] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:22:59.500674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d31cb7-de9c-4934-bb2d-af6788a6daff None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:59.513822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 518/1032] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:22:59 2022] POST /identity/v3/projects => generated 311 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:22:59.634227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d31cb7-de9c-4934-bb2d-af6788a6daff None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:59.656901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:59.675884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d413e-0442-4ffd-9b6f-5d80071bf0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:59.676095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d413e-0442-4ffd-9b6f-5d80071bf0e6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:59.676306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d413e-0442-4ffd-9b6f-5d80071bf0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:59.677561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d413e-0442-4ffd-9b6f-5d80071bf0e6 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:59.677561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d413e-0442-4ffd-9b6f-5d80071bf0e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:22:59.677561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d413e-0442-4ffd-9b6f-5d80071bf0e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:59.679970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d413e-0442-4ffd-9b6f-5d80071bf0e6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:59.692977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 515/1033] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:22:59.703003 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e237e-744c-45ac-98e2-2064e10fc379 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:22:59.773841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:22:59.850489 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e237e-744c-45ac-98e2-2064e10fc379 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:22:59.889897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab0d8-fb37-40bd-ad80-b4c10b663557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:22:59.890907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab0d8-fb37-40bd-ad80-b4c10b663557 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:22:59.891375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab0d8-fb37-40bd-ad80-b4c10b663557 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:22:59.891879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab0d8-fb37-40bd-ad80-b4c10b663557 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:22:59.901417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ab0d8-fb37-40bd-ad80-b4c10b663557 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:22:59.904221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ab0d8-fb37-40bd-ad80-b4c10b663557 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-ff5ab0d8-fb37-40bd-ad80-b4c10b663557 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:22:59.919492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:22:59.920517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:22:59.921477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:22:59.921477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:22:59.921477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:22:59.921477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:22:59.921477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:22:59.921477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:22:59.921477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:22:59.921477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:22:59.921477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:22:59.921477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:22:59.924161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 519/1034] 10.222.0.73 () {56 vars in 1093 bytes} [Thu Aug 4 21:22:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:22:59.930026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d413e-0442-4ffd-9b6f-5d80071bf0e6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:00.051880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d413e-0442-4ffd-9b6f-5d80071bf0e6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:00.091695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64f7e2-e320-4d8a-a71a-4f486dbdff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:00.092665 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64f7e2-e320-4d8a-a71a-4f486dbdff8a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:00.093124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64f7e2-e320-4d8a-a71a-4f486dbdff8a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:00.093570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64f7e2-e320-4d8a-a71a-4f486dbdff8a None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:00.094584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64f7e2-e320-4d8a-a71a-4f486dbdff8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:00.095189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64f7e2-e320-4d8a-a71a-4f486dbdff8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:00.096995 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64f7e2-e320-4d8a-a71a-4f486dbdff8a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:00.118391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 516/1035] 10.222.0.73 () {56 vars in 1133 bytes} [Thu Aug 4 21:22:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:00.123859 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab0d8-fb37-40bd-ad80-b4c10b663557 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:00.245388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab0d8-fb37-40bd-ad80-b4c10b663557 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:00.256442 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:00.294116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f3f4c-5fd3-4a55-9f15-64f49226822f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:00.295073 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f3f4c-5fd3-4a55-9f15-64f49226822f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:00.295538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f3f4c-5fd3-4a55-9f15-64f49226822f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:00.295993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f3f4c-5fd3-4a55-9f15-64f49226822f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:00.387801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f3f4c-5fd3-4a55-9f15-64f49226822f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:00.389600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f3f4c-5fd3-4a55-9f15-64f49226822f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:00.506080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 520/1036] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:00 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:23:00.511987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64f7e2-e320-4d8a-a71a-4f486dbdff8a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:00.561707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:00.629037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64f7e2-e320-4d8a-a71a-4f486dbdff8a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:00.666337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca630f2-4c41-44f8-b142-fadaf250fecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:00.666989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca630f2-4c41-44f8-b142-fadaf250fecd None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:00.667266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca630f2-4c41-44f8-b142-fadaf250fecd None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:00.667548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca630f2-4c41-44f8-b142-fadaf250fecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:00.678820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca630f2-4c41-44f8-b142-fadaf250fecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:00.680275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca630f2-4c41-44f8-b142-fadaf250fecd None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-6ca630f2-4c41-44f8-b142-fadaf250fecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:23:00.696063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:00.697040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:00.697946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:23:00.697946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:23:00.697946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:00.697946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:00.706074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 517/1037] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:00.706286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f3f4c-5fd3-4a55-9f15-64f49226822f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:00.716039 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:00.804870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920391d5-3a1d-4f51-a0ed-c1cee5572448 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:00.805171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920391d5-3a1d-4f51-a0ed-c1cee5572448 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:00.805354 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920391d5-3a1d-4f51-a0ed-c1cee5572448 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:00.806799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 518/1038] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:23:00.812116 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97499df-480a-4993-98dd-1882a26bb822 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:00.812403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97499df-480a-4993-98dd-1882a26bb822 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:00.812670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f3f4c-5fd3-4a55-9f15-64f49226822f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:00.812775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97499df-480a-4993-98dd-1882a26bb822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:00.832210 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-d97499df-480a-4993-98dd-1882a26bb822 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:00.833409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d97499df-480a-4993-98dd-1882a26bb822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:00.857563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245b74f-c584-4694-9950-e6072554d576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:00.858214 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245b74f-c584-4694-9950-e6072554d576 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:00.858501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245b74f-c584-4694-9950-e6072554d576 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:00.858772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245b74f-c584-4694-9950-e6072554d576 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:00.859461 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245b74f-c584-4694-9950-e6072554d576 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:00.859897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245b74f-c584-4694-9950-e6072554d576 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:00.862111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245b74f-c584-4694-9950-e6072554d576 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:00.874862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 521/1039] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:00.880847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245b74f-c584-4694-9950-e6072554d576 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:00.928115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 519/1040] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:23:00.949155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97499df-480a-4993-98dd-1882a26bb822 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:00.962344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:00.974510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:01.008730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245b74f-c584-4694-9950-e6072554d576 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:01.061788 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91270eb9-3f50-4bb1-9e8d-6ac4243ba888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:01.062394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91270eb9-3f50-4bb1-9e8d-6ac4243ba888 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:01.062677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91270eb9-3f50-4bb1-9e8d-6ac4243ba888 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:01.062949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91270eb9-3f50-4bb1-9e8d-6ac4243ba888 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:01.065479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97499df-480a-4993-98dd-1882a26bb822 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:01.071646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91270eb9-3f50-4bb1-9e8d-6ac4243ba888 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:01.073092 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91270eb9-3f50-4bb1-9e8d-6ac4243ba888 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-91270eb9-3f50-4bb1-9e8d-6ac4243ba888 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:23:01.086979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:01.087950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:01.088917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:23:01.088917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:23:01.088917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:23:01.088917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:23:01.088917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:23:01.088917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:23:01.088917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:23:01.088917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:23:01.093303 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 522/1041] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:23:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:23:01.094323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91270eb9-3f50-4bb1-9e8d-6ac4243ba888 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:01.109018 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa05a9c-ecdc-4b50-9f2d-d6a599694e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:01.109949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa05a9c-ecdc-4b50-9f2d-d6a599694e59 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:01.110434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa05a9c-ecdc-4b50-9f2d-d6a599694e59 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:01.110869 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa05a9c-ecdc-4b50-9f2d-d6a599694e59 None admin] PATH_INFO: `/v3/services/image` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:01.112183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa05a9c-ecdc-4b50-9f2d-d6a599694e59 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:01.114492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa05a9c-ecdc-4b50-9f2d-d6a599694e59 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-3fa05a9c-ecdc-4b50-9f2d-d6a599694e59 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:01.124786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:01.125899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:01.127805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 520/1042] 10.222.0.73 () {56 vars in 1105 bytes} [Thu Aug 4 21:23:00 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:23:01.135507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa05a9c-ecdc-4b50-9f2d-d6a599694e59 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:01.222421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91270eb9-3f50-4bb1-9e8d-6ac4243ba888 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:01.236861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:01.251696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:01.260318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d1587-69c7-4933-a7fc-1f2946f50e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:01.261009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d1587-69c7-4933-a7fc-1f2946f50e3c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:01.261207 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa05a9c-ecdc-4b50-9f2d-d6a599694e59 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:01.261308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d1587-69c7-4933-a7fc-1f2946f50e3c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:01.261500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d1587-69c7-4933-a7fc-1f2946f50e3c None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:01.262393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d1587-69c7-4933-a7fc-1f2946f50e3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:01.262628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d1587-69c7-4933-a7fc-1f2946f50e3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:01.264485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d1587-69c7-4933-a7fc-1f2946f50e3c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:01.285249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 523/1043] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:23:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:01.287988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d1587-69c7-4933-a7fc-1f2946f50e3c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:01.309147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d88a0f-44d7-44a8-8cc7-c753abe3d088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:01.310153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d88a0f-44d7-44a8-8cc7-c753abe3d088 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:01.310604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d88a0f-44d7-44a8-8cc7-c753abe3d088 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:01.311051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d88a0f-44d7-44a8-8cc7-c753abe3d088 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:01.312182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d88a0f-44d7-44a8-8cc7-c753abe3d088 None admin] RBAC: Adding query filter params (name=image) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:01.312599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d88a0f-44d7-44a8-8cc7-c753abe3d088 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:01.315697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d88a0f-44d7-44a8-8cc7-c753abe3d088 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:01.325158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 521/1044] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:01.331396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d88a0f-44d7-44a8-8cc7-c753abe3d088 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:01.420699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d1587-69c7-4933-a7fc-1f2946f50e3c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:01.458647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d88a0f-44d7-44a8-8cc7-c753abe3d088 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:01.461929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefe82b-c8d9-4220-81d1-f683d299e216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:01.462598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe82b-c8d9-4220-81d1-f683d299e216 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:01.462882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe82b-c8d9-4220-81d1-f683d299e216 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:01.463156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe82b-c8d9-4220-81d1-f683d299e216 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:01.464181 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefe82b-c8d9-4220-81d1-f683d299e216 None admin] RBAC: Adding query filter params (role.id=c8d5a540444e458fbccbcad1491b2872, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=e7b8267780364a2181abf5817b54ce5c) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:01.464295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefe82b-c8d9-4220-81d1-f683d299e216 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:01.465790 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefe82b-c8d9-4220-81d1-f683d299e216 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:01.482561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 524/1045] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:23:01 2022] GET /identity/v3/role_assignments?user.id=e7b8267780364a2181abf5817b54ce5c&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=c8d5a540444e458fbccbcad1491b2872 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:01.504446 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b51bd-1989-4aca-ad09-909f71cadef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:01.505102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b51bd-1989-4aca-ad09-909f71cadef8 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:01.505456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b51bd-1989-4aca-ad09-909f71cadef8 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:01.505737 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b51bd-1989-4aca-ad09-909f71cadef8 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:01.506480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b51bd-1989-4aca-ad09-909f71cadef8 None admin] RBAC: Adding query filter params (type=image) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:01.506784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b51bd-1989-4aca-ad09-909f71cadef8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:01.508179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b51bd-1989-4aca-ad09-909f71cadef8 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:01.519099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 522/1046] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:01.520505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefe82b-c8d9-4220-81d1-f683d299e216 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:01.590131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:01.649119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefe82b-c8d9-4220-81d1-f683d299e216 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:01.651403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6483b-0ceb-44a3-bf62-4bed5e82e47c None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:01.651699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6483b-0ceb-44a3-bf62-4bed5e82e47c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:01.651977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6483b-0ceb-44a3-bf62-4bed5e82e47c None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:01.653392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 523/1047] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:23:01.662807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d46acd-795a-4c7a-aa74-a0567546e745 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:01.663120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d46acd-795a-4c7a-aa74-a0567546e745 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:01.663403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d46acd-795a-4c7a-aa74-a0567546e745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:01.688153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-e5d46acd-795a-4c7a-aa74-a0567546e745 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:01.689952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e5d46acd-795a-4c7a-aa74-a0567546e745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:01.696022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bc880-b06a-4ec5-832f-0f72e1a217a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:01.697007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bc880-b06a-4ec5-832f-0f72e1a217a4 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:01.697501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bc880-b06a-4ec5-832f-0f72e1a217a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:01.697955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bc880-b06a-4ec5-832f-0f72e1a217a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:01.698898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410bc880-b06a-4ec5-832f-0f72e1a217a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:01.699707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:01.700625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410bc880-b06a-4ec5-832f-0f72e1a217a4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:01.766147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 524/1048] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:23:01.791981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d46acd-795a-4c7a-aa74-a0567546e745 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:01.843348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 525/1049] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:23:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:23:01.914572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d46acd-795a-4c7a-aa74-a0567546e745 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:02.029600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a9ccf-2cba-4a29-a74d-6937514035f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:02.030347 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a9ccf-2cba-4a29-a74d-6937514035f2 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:02.030589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a9ccf-2cba-4a29-a74d-6937514035f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:02.030897 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a9ccf-2cba-4a29-a74d-6937514035f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:02.032026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a9ccf-2cba-4a29-a74d-6937514035f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:02.034142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a9ccf-2cba-4a29-a74d-6937514035f2 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:02.036213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 525/1050] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:02.041929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bc880-b06a-4ec5-832f-0f72e1a217a4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:02.170277 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bc880-b06a-4ec5-832f-0f72e1a217a4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:02.179980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:02.205844 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448a1f6-f4c8-487f-9758-284f80f17115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:02.206507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448a1f6-f4c8-487f-9758-284f80f17115 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:02.206825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448a1f6-f4c8-487f-9758-284f80f17115 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:02.207105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448a1f6-f4c8-487f-9758-284f80f17115 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:02.207834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448a1f6-f4c8-487f-9758-284f80f17115 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:02.209295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448a1f6-f4c8-487f-9758-284f80f17115 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:02.303923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 526/1051] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:23:02 2022] POST /identity/v3/users => generated 285 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:23:03.600501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84549bb-0fa6-43cf-bb7c-a907b5473678 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:03.601097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84549bb-0fa6-43cf-bb7c-a907b5473678 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:03.601097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84549bb-0fa6-43cf-bb7c-a907b5473678 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:03.602614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 526/1052] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:03.607986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71c276-ff88-4431-b759-31641e7b8ad0 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:03.608418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71c276-ff88-4431-b759-31641e7b8ad0 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:03.608592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71c276-ff88-4431-b759-31641e7b8ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:03.629338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-bb71c276-ff88-4431-b759-31641e7b8ad0 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:03.630607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bb71c276-ff88-4431-b759-31641e7b8ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:03.706568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 527/1053] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:03.730055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84549bb-0fa6-43cf-bb7c-a907b5473678 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:03.740681 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:03.869664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84549bb-0fa6-43cf-bb7c-a907b5473678 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:03.913099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f99763-566c-4799-a13c-d474cf6498ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:03.913738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f99763-566c-4799-a13c-d474cf6498ad None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:03.914034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f99763-566c-4799-a13c-d474cf6498ad None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:03.914452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f99763-566c-4799-a13c-d474cf6498ad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:03.927103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f99763-566c-4799-a13c-d474cf6498ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-06f99763-566c-4799-a13c-d474cf6498ad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:23:03.932304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:03.933414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:23:03.934303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:03.938884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 527/1054] 10.222.0.73 () {56 vars in 1120 bytes} [Thu Aug 4 21:23:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:03.940432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71c276-ff88-4431-b759-31641e7b8ad0 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:03.985463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda911e-0e6f-4cd9-b780-4e58ffd9972b None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:03.985768 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda911e-0e6f-4cd9-b780-4e58ffd9972b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:03.986052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda911e-0e6f-4cd9-b780-4e58ffd9972b None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:03.988038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 528/1055] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:23:03.994216 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3529f-7e8b-4a9c-a040-94f797d66762 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:03.994580 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3529f-7e8b-4a9c-a040-94f797d66762 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:03.994814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3529f-7e8b-4a9c-a040-94f797d66762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.018926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-34d3529f-7e8b-4a9c-a040-94f797d66762 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:04.020541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-34d3529f-7e8b-4a9c-a040-94f797d66762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:04.072552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:04.079887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71c276-ff88-4431-b759-31641e7b8ad0 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:04.100333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:04.109166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 529/1056] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:04.120065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfcbf3-58d5-4af4-a331-75d0999e30de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:04.120668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfcbf3-58d5-4af4-a331-75d0999e30de None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.120952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfcbf3-58d5-4af4-a331-75d0999e30de None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.121225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfcbf3-58d5-4af4-a331-75d0999e30de None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.121947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfcbf3-58d5-4af4-a331-75d0999e30de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:04.122281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfcbf3-58d5-4af4-a331-75d0999e30de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:04.123669 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfcbf3-58d5-4af4-a331-75d0999e30de None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:04.132864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 528/1057] 10.222.0.73 () {56 vars in 1115 bytes} [Thu Aug 4 21:23:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:04.134414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3529f-7e8b-4a9c-a040-94f797d66762 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:04.138453 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfcbf3-58d5-4af4-a331-75d0999e30de None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:04.255998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3529f-7e8b-4a9c-a040-94f797d66762 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:04.261898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfcbf3-58d5-4af4-a331-75d0999e30de None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:04.295486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c088558-4210-4bcb-af38-52f178a4512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:04.296203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c088558-4210-4bcb-af38-52f178a4512b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.296549 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c088558-4210-4bcb-af38-52f178a4512b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.296843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c088558-4210-4bcb-af38-52f178a4512b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.306000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f85d46-6648-453f-aa11-2b8e5ad4043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:04.306999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f85d46-6648-453f-aa11-2b8e5ad4043b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.307594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f85d46-6648-453f-aa11-2b8e5ad4043b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.308031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f85d46-6648-453f-aa11-2b8e5ad4043b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.323789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f85d46-6648-453f-aa11-2b8e5ad4043b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:04.325997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f85d46-6648-453f-aa11-2b8e5ad4043b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-46f85d46-6648-453f-aa11-2b8e5ad4043b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:23:04.345564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:04.347026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:04.347964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:23:04.347964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:23:04.347964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:04.347964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:23:04.350451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 529/1058] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:23:04.353989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f85d46-6648-453f-aa11-2b8e5ad4043b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:04.374906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:04.384225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:04.392469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c088558-4210-4bcb-af38-52f178a4512b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:04.394541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c088558-4210-4bcb-af38-52f178a4512b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:04.495114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f85d46-6648-453f-aa11-2b8e5ad4043b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:04.520841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 530/1059] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:23:04.520841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0814a5-5998-4207-9d5a-98ab43371bf0 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.520841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0814a5-5998-4207-9d5a-98ab43371bf0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.520841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0814a5-5998-4207-9d5a-98ab43371bf0 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.520841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 531/1060] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:04.527129 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0814a5-5998-4207-9d5a-98ab43371bf0 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:04.540407 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861030a-0664-4855-9925-4d33d098828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:04.541028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861030a-0664-4855-9925-4d33d098828f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.541308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861030a-0664-4855-9925-4d33d098828f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.541580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861030a-0664-4855-9925-4d33d098828f None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.542305 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861030a-0664-4855-9925-4d33d098828f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:04.542606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861030a-0664-4855-9925-4d33d098828f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:04.543988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861030a-0664-4855-9925-4d33d098828f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:04.557882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 530/1061] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:04.559216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec928f-4f54-4171-9d05-557b60cdaac7 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.559504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec928f-4f54-4171-9d05-557b60cdaac7 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.559780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec928f-4f54-4171-9d05-557b60cdaac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.578933 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a4ec928f-4f54-4171-9d05-557b60cdaac7 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:04.580485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a4ec928f-4f54-4171-9d05-557b60cdaac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:04.645328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:04.647630 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:04.652200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 531/1062] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:04.655702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0814a5-5998-4207-9d5a-98ab43371bf0 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:04.656734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec928f-4f54-4171-9d05-557b60cdaac7 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:04.693843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48424e-8c49-4cb2-98ef-452e7cc54690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:04.695094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48424e-8c49-4cb2-98ef-452e7cc54690 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.695377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48424e-8c49-4cb2-98ef-452e7cc54690 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.695654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48424e-8c49-4cb2-98ef-452e7cc54690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.706367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48424e-8c49-4cb2-98ef-452e7cc54690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:04.707813 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48424e-8c49-4cb2-98ef-452e7cc54690 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-4d48424e-8c49-4cb2-98ef-452e7cc54690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:23:04.723977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:04.725204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:04.726297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:23:04.726297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:23:04.726297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:04.726297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:04.726297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 532/1063] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:04.726297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48424e-8c49-4cb2-98ef-452e7cc54690 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:04.763105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec928f-4f54-4171-9d05-557b60cdaac7 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:04.803814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c9baa-3b0c-4a22-b14c-5fcb394c8553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:04.804495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c9baa-3b0c-4a22-b14c-5fcb394c8553 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.804867 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c9baa-3b0c-4a22-b14c-5fcb394c8553 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.805151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c9baa-3b0c-4a22-b14c-5fcb394c8553 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.813386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c9baa-3b0c-4a22-b14c-5fcb394c8553 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:04.814997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c9baa-3b0c-4a22-b14c-5fcb394c8553 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-5b4c9baa-3b0c-4a22-b14c-5fcb394c8553 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:23:04.828198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:04.829509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 532/1064] 10.222.0.73 () {56 vars in 1093 bytes} [Thu Aug 4 21:23:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:04.830592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c9baa-3b0c-4a22-b14c-5fcb394c8553 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:04.841373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48424e-8c49-4cb2-98ef-452e7cc54690 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:04.882921 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bd136-94a6-4646-bf98-61925a19d70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:04.883573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd136-94a6-4646-bf98-61925a19d70a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.883877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd136-94a6-4646-bf98-61925a19d70a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.884127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd136-94a6-4646-bf98-61925a19d70a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.892080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:04.896644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709bd136-94a6-4646-bf98-61925a19d70a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-709bd136-94a6-4646-bf98-61925a19d70a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:23:04.901762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:04.909750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 533/1065] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:23:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:04.910638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bd136-94a6-4646-bf98-61925a19d70a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:04.919822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:04.936501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c9baa-3b0c-4a22-b14c-5fcb394c8553 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:04.973407 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687cc96-8652-4203-b825-23bef2905c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:04.974053 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687cc96-8652-4203-b825-23bef2905c3f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:04.974343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687cc96-8652-4203-b825-23bef2905c3f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:04.974624 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687cc96-8652-4203-b825-23bef2905c3f None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:04.975326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687cc96-8652-4203-b825-23bef2905c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:04.975634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687cc96-8652-4203-b825-23bef2905c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:04.977027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687cc96-8652-4203-b825-23bef2905c3f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:04.991129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 533/1066] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:04.991991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687cc96-8652-4203-b825-23bef2905c3f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:05.031456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bd136-94a6-4646-bf98-61925a19d70a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:05.076867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217d574-af02-454a-89a5-050da947e4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:05.077497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217d574-af02-454a-89a5-050da947e4dd None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.077849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217d574-af02-454a-89a5-050da947e4dd None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.078253 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217d574-af02-454a-89a5-050da947e4dd None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.079803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217d574-af02-454a-89a5-050da947e4dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:05.080171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217d574-af02-454a-89a5-050da947e4dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:05.081668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217d574-af02-454a-89a5-050da947e4dd None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:05.100759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 534/1067] 10.222.0.73 () {56 vars in 1133 bytes} [Thu Aug 4 21:23:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:05.106726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217d574-af02-454a-89a5-050da947e4dd None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:05.132222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687cc96-8652-4203-b825-23bef2905c3f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:05.155167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:05.178954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01314e79-6639-455a-a2b8-571e811b77ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:05.179602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01314e79-6639-455a-a2b8-571e811b77ec None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.180352 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01314e79-6639-455a-a2b8-571e811b77ec None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.180644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01314e79-6639-455a-a2b8-571e811b77ec None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.181491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01314e79-6639-455a-a2b8-571e811b77ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:05.182061 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01314e79-6639-455a-a2b8-571e811b77ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:05.183941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01314e79-6639-455a-a2b8-571e811b77ec None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:05.198583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 534/1068] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:23:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:05.203180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:05.205457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01314e79-6639-455a-a2b8-571e811b77ec None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:05.247198 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217d574-af02-454a-89a5-050da947e4dd None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:05.294921 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c91f1-5d59-49e4-8588-e439279c8057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:05.295593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c91f1-5d59-49e4-8588-e439279c8057 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.295876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c91f1-5d59-49e4-8588-e439279c8057 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.296175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c91f1-5d59-49e4-8588-e439279c8057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.311614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c91f1-5d59-49e4-8588-e439279c8057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:05.313863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c91f1-5d59-49e4-8588-e439279c8057 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-ca6c91f1-5d59-49e4-8588-e439279c8057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:23:05.332807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:05.333933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:05.335132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:23:05.335132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:23:05.335132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:05.335132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:05.335132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 535/1069] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:23:05.339783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c91f1-5d59-49e4-8588-e439279c8057 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:05.346259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01314e79-6639-455a-a2b8-571e811b77ec None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:05.394023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf95bc3-1048-4355-b8db-d6d03a20ac33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:05.394874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf95bc3-1048-4355-b8db-d6d03a20ac33 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.395188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf95bc3-1048-4355-b8db-d6d03a20ac33 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.395505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf95bc3-1048-4355-b8db-d6d03a20ac33 None admin] PATH_INFO: `/v3/users/3d07c5ee849548f4ac182934a84b2236` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.413795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf95bc3-1048-4355-b8db-d6d03a20ac33 None admin] RBAC: Authorizing `identity:get_user(user_id=3d07c5ee849548f4ac182934a84b2236)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:05.415678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf95bc3-1048-4355-b8db-d6d03a20ac33 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:05.418129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 535/1070] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:05 2022] GET /identity/v3/users/3d07c5ee849548f4ac182934a84b2236 => generated 285 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:05.424851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf95bc3-1048-4355-b8db-d6d03a20ac33 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:05.474454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c91f1-5d59-49e4-8588-e439279c8057 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:05.479587 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:05.498191 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:05.515238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e11c1-7fd6-4499-8b19-a9bd337ab825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:05.515858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e11c1-7fd6-4499-8b19-a9bd337ab825 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.516134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e11c1-7fd6-4499-8b19-a9bd337ab825 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.516403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e11c1-7fd6-4499-8b19-a9bd337ab825 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.517786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e11c1-7fd6-4499-8b19-a9bd337ab825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:05.518147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e11c1-7fd6-4499-8b19-a9bd337ab825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:05.519508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e11c1-7fd6-4499-8b19-a9bd337ab825 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:05.536064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 536/1071] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:05.542676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e11c1-7fd6-4499-8b19-a9bd337ab825 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:05.546332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf95bc3-1048-4355-b8db-d6d03a20ac33 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:05.592093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62f81-b4ec-436e-80e5-5d13ad1e4f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:05.592738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62f81-b4ec-436e-80e5-5d13ad1e4f11 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.593058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62f81-b4ec-436e-80e5-5d13ad1e4f11 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.593379 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62f81-b4ec-436e-80e5-5d13ad1e4f11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.644368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e11c1-7fd6-4499-8b19-a9bd337ab825 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:05.666394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e62f81-b4ec-436e-80e5-5d13ad1e4f11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:05.667766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e62f81-b4ec-436e-80e5-5d13ad1e4f11 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:05.678731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c19d7-9f8c-43a9-8c1a-bca8c0542605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:05.679378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c19d7-9f8c-43a9-8c1a-bca8c0542605 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.679664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c19d7-9f8c-43a9-8c1a-bca8c0542605 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.679940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c19d7-9f8c-43a9-8c1a-bca8c0542605 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.686914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c19d7-9f8c-43a9-8c1a-bca8c0542605 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:05.688384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c19d7-9f8c-43a9-8c1a-bca8c0542605 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-186c19d7-9f8c-43a9-8c1a-bca8c0542605 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:23:05.700933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:05.702097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:05.703044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:23:05.703044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:23:05.703044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:23:05.703044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:23:05.703044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:23:05.703044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:23:05.703044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:23:05.703044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:05.703044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 537/1072] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:23:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:23:05.708725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c19d7-9f8c-43a9-8c1a-bca8c0542605 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:05.710445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:05.749946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:05.773540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 536/1073] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 21:23:05.779558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62f81-b4ec-436e-80e5-5d13ad1e4f11 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:05.823555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c19d7-9f8c-43a9-8c1a-bca8c0542605 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:05.858472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a6efb-13f7-4abc-a6c7-38d1ef2cf144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:05.859103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a6efb-13f7-4abc-a6c7-38d1ef2cf144 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.859384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a6efb-13f7-4abc-a6c7-38d1ef2cf144 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.859660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a6efb-13f7-4abc-a6c7-38d1ef2cf144 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.860966 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a6efb-13f7-4abc-a6c7-38d1ef2cf144 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:05.861302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a6efb-13f7-4abc-a6c7-38d1ef2cf144 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:05.863027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a6efb-13f7-4abc-a6c7-38d1ef2cf144 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:05.874977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 538/1074] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:23:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:05.882789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a6efb-13f7-4abc-a6c7-38d1ef2cf144 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:05.886638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62f81-b4ec-436e-80e5-5d13ad1e4f11 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:05.922632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c362400-94e7-4732-9d84-385139e344b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:05.923761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c362400-94e7-4732-9d84-385139e344b3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:05.924117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c362400-94e7-4732-9d84-385139e344b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:05.924404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c362400-94e7-4732-9d84-385139e344b3 None admin] PATH_INFO: `/v3/projects/11b97170740f493fa15e6bf7b0a27dd4` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:05.925574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c362400-94e7-4732-9d84-385139e344b3 None admin] RBAC: Authorizing `identity:get_project(project_id=11b97170740f493fa15e6bf7b0a27dd4)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:05.927099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c362400-94e7-4732-9d84-385139e344b3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:05.928759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 537/1075] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:05 2022] GET /identity/v3/projects/11b97170740f493fa15e6bf7b0a27dd4 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:05.935005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c362400-94e7-4732-9d84-385139e344b3 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:05.946213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:05.982979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:05.987585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a6efb-13f7-4abc-a6c7-38d1ef2cf144 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:06.021755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35364377-a1ac-49d1-b3ac-a24fe3211fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:06.022506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35364377-a1ac-49d1-b3ac-a24fe3211fb0 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:06.022788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35364377-a1ac-49d1-b3ac-a24fe3211fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:06.023118 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35364377-a1ac-49d1-b3ac-a24fe3211fb0 None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/e7b8267780364a2181abf5817b54ce5c/roles/c8d5a540444e458fbccbcad1491b2872` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:06.025348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35364377-a1ac-49d1-b3ac-a24fe3211fb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=e7b8267780364a2181abf5817b54ce5c, role_id=c8d5a540444e458fbccbcad1491b2872)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:06.026814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35364377-a1ac-49d1-b3ac-a24fe3211fb0 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:06.037653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c362400-94e7-4732-9d84-385139e344b3 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:06.073417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd656c-b919-404f-b988-027d0ed849c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:06.074108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd656c-b919-404f-b988-027d0ed849c1 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:06.074398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd656c-b919-404f-b988-027d0ed849c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:06.074673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd656c-b919-404f-b988-027d0ed849c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:06.075459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd656c-b919-404f-b988-027d0ed849c1 None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=11b97170740f493fa15e6bf7b0a27dd4, user.id=3d07c5ee849548f4ac182934a84b2236) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:06.075770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd656c-b919-404f-b988-027d0ed849c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:06.077201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd656c-b919-404f-b988-027d0ed849c1 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:06.082695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 538/1076] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:23:05 2022] GET /identity/v3/role_assignments?user.id=3d07c5ee849548f4ac182934a84b2236&scope.project.id=11b97170740f493fa15e6bf7b0a27dd4&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:06.123830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 539/1077] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:23:05 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/e7b8267780364a2181abf5817b54ce5c/roles/c8d5a540444e458fbccbcad1491b2872 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:23:08.156776 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f63da-9748-42e6-9952-b935a45492c4 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.157344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f63da-9748-42e6-9952-b935a45492c4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.157344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f63da-9748-42e6-9952-b935a45492c4 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.158692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 539/1078] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:08.164797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dd13f-e4fa-4407-89b3-7f4465cec2b7 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.165271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dd13f-e4fa-4407-89b3-7f4465cec2b7 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.165615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dd13f-e4fa-4407-89b3-7f4465cec2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.166834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2acf18-aeb1-4cb5-a1f3-3cd071f0dc25 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.167118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2acf18-aeb1-4cb5-a1f3-3cd071f0dc25 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.167391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2acf18-aeb1-4cb5-a1f3-3cd071f0dc25 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.168659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 540/1079] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:23:08.173667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3413ed-da42-43fd-8e1a-7564c5e994c8 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.173908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3413ed-da42-43fd-8e1a-7564c5e994c8 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.174228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3413ed-da42-43fd-8e1a-7564c5e994c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.184668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-879dd13f-e4fa-4407-89b3-7f4465cec2b7 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:08.185857 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-879dd13f-e4fa-4407-89b3-7f4465cec2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:08.191628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-cb3413ed-da42-43fd-8e1a-7564c5e994c8 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:08.192791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-cb3413ed-da42-43fd-8e1a-7564c5e994c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:08.253903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 540/1080] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:08.257326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:08.262467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 541/1081] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:23:08.278433 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879dd13f-e4fa-4407-89b3-7f4465cec2b7 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:08.286454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3413ed-da42-43fd-8e1a-7564c5e994c8 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:08.370210 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:08.381861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879dd13f-e4fa-4407-89b3-7f4465cec2b7 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:08.388473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3413ed-da42-43fd-8e1a-7564c5e994c8 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:08.419015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d7c47-fff3-483b-aed7-a6d451e64f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:08.419585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d7c47-fff3-483b-aed7-a6d451e64f47 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.419872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d7c47-fff3-483b-aed7-a6d451e64f47 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.420148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d7c47-fff3-483b-aed7-a6d451e64f47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.427376 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3db23-da8f-483c-a75c-c5be744573b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:08.428017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3db23-da8f-483c-a75c-c5be744573b2 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.428298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3db23-da8f-483c-a75c-c5be744573b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.428572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3db23-da8f-483c-a75c-c5be744573b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.430548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d7c47-fff3-483b-aed7-a6d451e64f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-441d7c47-fff3-483b-aed7-a6d451e64f47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:23:08.435545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:08.436547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:08.437412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 541/1082] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:23:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:08.439669 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3db23-da8f-483c-a75c-c5be744573b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:23:08.443415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d7c47-fff3-483b-aed7-a6d451e64f47 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-69e3db23-da8f-483c-a75c-c5be744573b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:23:08.444549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:08.445646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:23:08.446563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 542/1083] 10.222.0.73 () {56 vars in 1120 bytes} [Thu Aug 4 21:23:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:23:08.451639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3db23-da8f-483c-a75c-c5be744573b2 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:08.512344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:08.550261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d7c47-fff3-483b-aed7-a6d451e64f47 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:08.554369 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3db23-da8f-483c-a75c-c5be744573b2 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:08.591312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c575c-3802-4811-ae19-42bcfac33388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:08.591827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12067fe1-38e8-4dae-bcd6-73c852a7afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:08.591951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c575c-3802-4811-ae19-42bcfac33388 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.592203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c575c-3802-4811-ae19-42bcfac33388 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.592449 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12067fe1-38e8-4dae-bcd6-73c852a7afb5 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.592558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c575c-3802-4811-ae19-42bcfac33388 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.592732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12067fe1-38e8-4dae-bcd6-73c852a7afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.593010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12067fe1-38e8-4dae-bcd6-73c852a7afb5 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.593177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c575c-3802-4811-ae19-42bcfac33388 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:08.593481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c575c-3802-4811-ae19-42bcfac33388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:08.593722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12067fe1-38e8-4dae-bcd6-73c852a7afb5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:08.594052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12067fe1-38e8-4dae-bcd6-73c852a7afb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:08.594969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c575c-3802-4811-ae19-42bcfac33388 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:08.595815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12067fe1-38e8-4dae-bcd6-73c852a7afb5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:08.603584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 542/1084] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:23:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:08.604316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 543/1085] 10.222.0.73 () {56 vars in 1115 bytes} [Thu Aug 4 21:23:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:08.610323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c575c-3802-4811-ae19-42bcfac33388 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:08.610520 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12067fe1-38e8-4dae-bcd6-73c852a7afb5 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:08.624727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:08.714103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12067fe1-38e8-4dae-bcd6-73c852a7afb5 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:08.716710 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c575c-3802-4811-ae19-42bcfac33388 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:08.737379 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:08.749075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c74c8-b9d0-42cd-a4d9-3bef7e738b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:08.749711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c74c8-b9d0-42cd-a4d9-3bef7e738b97 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.754112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6c6ed-447b-4c81-808b-1f5430607dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:08.754255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c74c8-b9d0-42cd-a4d9-3bef7e738b97 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.754255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c74c8-b9d0-42cd-a4d9-3bef7e738b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.754411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6c6ed-447b-4c81-808b-1f5430607dbc None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.754411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6c6ed-447b-4c81-808b-1f5430607dbc None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.754411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6c6ed-447b-4c81-808b-1f5430607dbc None admin] PATH_INFO: `/v3/users/3d07c5ee849548f4ac182934a84b2236` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.754411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6c6ed-447b-4c81-808b-1f5430607dbc None admin] RBAC: Authorizing `identity:get_user(user_id=3d07c5ee849548f4ac182934a84b2236)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:08.754733 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6c6ed-447b-4c81-808b-1f5430607dbc None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:08.756280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 543/1086] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:08 2022] GET /identity/v3/users/3d07c5ee849548f4ac182934a84b2236 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:08.761392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c74c8-b9d0-42cd-a4d9-3bef7e738b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:08.762882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c74c8-b9d0-42cd-a4d9-3bef7e738b97 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:08.768587 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6c6ed-447b-4c81-808b-1f5430607dbc None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-450c74c8-b9d0-42cd-a4d9-3bef7e738b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:23:08.779266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:08.780208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:08.781113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:23:08.781113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:23:08.781113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:08.781113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:23:08.781113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 544/1087] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:23:08.786893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c74c8-b9d0-42cd-a4d9-3bef7e738b97 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:08.875345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:08.889402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c74c8-b9d0-42cd-a4d9-3bef7e738b97 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:08.910033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6c6ed-447b-4c81-808b-1f5430607dbc None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:08.933011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67ba84-0361-4dfb-bc76-43e888521135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:08.933632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67ba84-0361-4dfb-bc76-43e888521135 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.933912 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67ba84-0361-4dfb-bc76-43e888521135 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.934217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67ba84-0361-4dfb-bc76-43e888521135 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.934905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67ba84-0361-4dfb-bc76-43e888521135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:08.935208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67ba84-0361-4dfb-bc76-43e888521135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:08.936620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67ba84-0361-4dfb-bc76-43e888521135 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:08.945993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee10309-1848-42d2-bd6f-746ae0d52207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:08.946827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee10309-1848-42d2-bd6f-746ae0d52207 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:08.947117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee10309-1848-42d2-bd6f-746ae0d52207 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:08.947396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee10309-1848-42d2-bd6f-746ae0d52207 None admin] PATH_INFO: `/v3/projects/11b97170740f493fa15e6bf7b0a27dd4` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:08.948577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee10309-1848-42d2-bd6f-746ae0d52207 None admin] RBAC: Authorizing `identity:get_project(project_id=11b97170740f493fa15e6bf7b0a27dd4)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:08.949008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 545/1088] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:08.950146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee10309-1848-42d2-bd6f-746ae0d52207 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:08.951832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 544/1089] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:08 2022] GET /identity/v3/projects/11b97170740f493fa15e6bf7b0a27dd4 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:08.956990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67ba84-0361-4dfb-bc76-43e888521135 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:08.959015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee10309-1848-42d2-bd6f-746ae0d52207 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:08.995562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:09.069784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67ba84-0361-4dfb-bc76-43e888521135 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:09.078836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee10309-1848-42d2-bd6f-746ae0d52207 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:09.106474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f77e92-c915-4726-b3e7-b0f19e23e369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:09.107471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f77e92-c915-4726-b3e7-b0f19e23e369 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:09.107766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f77e92-c915-4726-b3e7-b0f19e23e369 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:09.108060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f77e92-c915-4726-b3e7-b0f19e23e369 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:09.116550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f77e92-c915-4726-b3e7-b0f19e23e369 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:09.117852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf8c50-e1a8-43f1-a0f7-e98f3bc89233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:09.118206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f77e92-c915-4726-b3e7-b0f19e23e369 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:09.118950 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf8c50-e1a8-43f1-a0f7-e98f3bc89233 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:09.119411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf8c50-e1a8-43f1-a0f7-e98f3bc89233 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:09.119864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf8c50-e1a8-43f1-a0f7-e98f3bc89233 None admin] PATH_INFO: `/v3/projects/11b97170740f493fa15e6bf7b0a27dd4/users/3d07c5ee849548f4ac182934a84b2236/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:09.122826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf8c50-e1a8-43f1-a0f7-e98f3bc89233 None admin] RBAC: Authorizing `identity:create_grant(project_id=11b97170740f493fa15e6bf7b0a27dd4, user_id=3d07c5ee849548f4ac182934a84b2236, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:09.125227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf8c50-e1a8-43f1-a0f7-e98f3bc89233 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-f6f77e92-c915-4726-b3e7-b0f19e23e369 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:23:09.131323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:09.132546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:09.133782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:09.133782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:09.133782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:23:09.133782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:23:09.133782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:23:09.133782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:23:09.133782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:23:09.133782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:23:09.133782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 21:23:09.133782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:23:09.133782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 546/1090] 10.222.0.73 () {56 vars in 1093 bytes} [Thu Aug 4 21:23:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:09.138447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f77e92-c915-4726-b3e7-b0f19e23e369 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:09.202832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 545/1091] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:23:08 2022] PUT /identity/v3/projects/11b97170740f493fa15e6bf7b0a27dd4/users/3d07c5ee849548f4ac182934a84b2236/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:23:09.250377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:09.258844 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f77e92-c915-4726-b3e7-b0f19e23e369 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:09.297430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187025b1-7f85-4a72-abea-c3b215229ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:09.298158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187025b1-7f85-4a72-abea-c3b215229ac8 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:09.298451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187025b1-7f85-4a72-abea-c3b215229ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:09.299100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187025b1-7f85-4a72-abea-c3b215229ac8 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:09.300174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187025b1-7f85-4a72-abea-c3b215229ac8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:09.300799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187025b1-7f85-4a72-abea-c3b215229ac8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:09.302852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187025b1-7f85-4a72-abea-c3b215229ac8 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:09.323017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 547/1092] 10.222.0.73 () {56 vars in 1133 bytes} [Thu Aug 4 21:23:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:09.329668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf8c50-e1a8-43f1-a0f7-e98f3bc89233 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:09.339431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:09.436036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf8c50-e1a8-43f1-a0f7-e98f3bc89233 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:09.471816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fea319-cb37-4b3a-8c4b-f0b85fb2bb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:09.472442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fea319-cb37-4b3a-8c4b-f0b85fb2bb27 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:09.472722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fea319-cb37-4b3a-8c4b-f0b85fb2bb27 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:09.472994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fea319-cb37-4b3a-8c4b-f0b85fb2bb27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:09.553748 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fea319-cb37-4b3a-8c4b-f0b85fb2bb27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:09.555364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fea319-cb37-4b3a-8c4b-f0b85fb2bb27 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:09.564252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:09.683046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 546/1093] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 21:23:09.684417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187025b1-7f85-4a72-abea-c3b215229ac8 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:09.785637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187025b1-7f85-4a72-abea-c3b215229ac8 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:09.820546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f27f6-6b81-437c-ae12-7863ec36f79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:09.821185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f27f6-6b81-437c-ae12-7863ec36f79b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:09.821463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f27f6-6b81-437c-ae12-7863ec36f79b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:09.821735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f27f6-6b81-437c-ae12-7863ec36f79b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:09.832015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f27f6-6b81-437c-ae12-7863ec36f79b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:09.833471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f27f6-6b81-437c-ae12-7863ec36f79b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-d52f27f6-6b81-437c-ae12-7863ec36f79b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:23:09.848969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:09.849951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:09.850999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:23:09.850999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:23:09.850999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:23:09.850999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:23:09.850999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 548/1094] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:09.856132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fea319-cb37-4b3a-8c4b-f0b85fb2bb27 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:09.958408 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fea319-cb37-4b3a-8c4b-f0b85fb2bb27 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:09.966993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:09.994651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222407c-efb9-42ce-b6ba-a8535e45bc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:09.995268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222407c-efb9-42ce-b6ba-a8535e45bc55 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:09.995545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222407c-efb9-42ce-b6ba-a8535e45bc55 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:09.996190 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222407c-efb9-42ce-b6ba-a8535e45bc55 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:09.997056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222407c-efb9-42ce-b6ba-a8535e45bc55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:09.997361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222407c-efb9-42ce-b6ba-a8535e45bc55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:09.998770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222407c-efb9-42ce-b6ba-a8535e45bc55 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:10.012141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 547/1095] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:10.018652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f27f6-6b81-437c-ae12-7863ec36f79b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:10.023157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:10.120703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f27f6-6b81-437c-ae12-7863ec36f79b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:10.155588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50383-c4ec-4666-8fc5-9bc235c2a061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:10.156239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50383-c4ec-4666-8fc5-9bc235c2a061 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:10.156521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50383-c4ec-4666-8fc5-9bc235c2a061 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:10.156794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50383-c4ec-4666-8fc5-9bc235c2a061 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:10.164070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb50383-c4ec-4666-8fc5-9bc235c2a061 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:10.165529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb50383-c4ec-4666-8fc5-9bc235c2a061 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-dbb50383-c4ec-4666-8fc5-9bc235c2a061 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:23:10.177343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:10.178547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:10.179536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:23:10.179536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:23:10.179536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:23:10.179536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:23:10.179536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:23:10.179536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:23:10.179536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:23:10.179536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:23:10.179536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 549/1096] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:23:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:10.184747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222407c-efb9-42ce-b6ba-a8535e45bc55 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:10.287190 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222407c-efb9-42ce-b6ba-a8535e45bc55 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:10.322050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e611d8f-bf73-47e5-885b-54b192ed49f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:10.322700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e611d8f-bf73-47e5-885b-54b192ed49f7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:10.322981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e611d8f-bf73-47e5-885b-54b192ed49f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:10.323254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e611d8f-bf73-47e5-885b-54b192ed49f7 None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:10.324034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e611d8f-bf73-47e5-885b-54b192ed49f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:10.324338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e611d8f-bf73-47e5-885b-54b192ed49f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:10.325811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e611d8f-bf73-47e5-885b-54b192ed49f7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:10.339727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 548/1097] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:23:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:10.346904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50383-c4ec-4666-8fc5-9bc235c2a061 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:10.412492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:10.451805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50383-c4ec-4666-8fc5-9bc235c2a061 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:10.488422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b81cc2-0ae5-4a13-a14d-2644a1dbf6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:10.489035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b81cc2-0ae5-4a13-a14d-2644a1dbf6be None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:10.489319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b81cc2-0ae5-4a13-a14d-2644a1dbf6be None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:10.489602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b81cc2-0ae5-4a13-a14d-2644a1dbf6be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:10.490454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b81cc2-0ae5-4a13-a14d-2644a1dbf6be None admin] RBAC: Adding query filter params (role.id=c8d5a540444e458fbccbcad1491b2872, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=e7b8267780364a2181abf5817b54ce5c) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:10.490766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b81cc2-0ae5-4a13-a14d-2644a1dbf6be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:10.492212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b81cc2-0ae5-4a13-a14d-2644a1dbf6be None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:10.510033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 550/1098] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:23:10 2022] GET /identity/v3/role_assignments?user.id=e7b8267780364a2181abf5817b54ce5c&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=c8d5a540444e458fbccbcad1491b2872 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:11.322361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65541038-8287-48af-a8bf-1593be4fc790 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:11.322888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65541038-8287-48af-a8bf-1593be4fc790 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:11.322975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65541038-8287-48af-a8bf-1593be4fc790 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:11.324573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 549/1099] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:11.330837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75a45e-8def-4be9-bd57-4dd12c662d57 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:11.331141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75a45e-8def-4be9-bd57-4dd12c662d57 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:11.331445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75a45e-8def-4be9-bd57-4dd12c662d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:11.351421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ee75a45e-8def-4be9-bd57-4dd12c662d57 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:11.352727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ee75a45e-8def-4be9-bd57-4dd12c662d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:11.424994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 551/1100] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:11.454578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65541038-8287-48af-a8bf-1593be4fc790 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:11.487131 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:11.567261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65541038-8287-48af-a8bf-1593be4fc790 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:11.601520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a195bb-8e0d-4d20-9fe4-d023498face4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:11.602099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a195bb-8e0d-4d20-9fe4-d023498face4 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:11.602383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a195bb-8e0d-4d20-9fe4-d023498face4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:11.602657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a195bb-8e0d-4d20-9fe4-d023498face4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:11.614105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a195bb-8e0d-4d20-9fe4-d023498face4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-48a195bb-8e0d-4d20-9fe4-d023498face4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:23:11.618929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:11.620576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:11.622055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 550/1101] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:23:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:11.626960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75a45e-8def-4be9-bd57-4dd12c662d57 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:11.692136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:11.733259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75a45e-8def-4be9-bd57-4dd12c662d57 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:11.768657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e1dad-7727-4407-b080-d6691d408921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:11.769302 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e1dad-7727-4407-b080-d6691d408921 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:11.769583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e1dad-7727-4407-b080-d6691d408921 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:11.769858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e1dad-7727-4407-b080-d6691d408921 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:11.770599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e1dad-7727-4407-b080-d6691d408921 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:11.770909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e1dad-7727-4407-b080-d6691d408921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:11.772311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e1dad-7727-4407-b080-d6691d408921 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:11.780900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 552/1102] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:23:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:11.786942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a195bb-8e0d-4d20-9fe4-d023498face4 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:11.875262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:11.892694 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a195bb-8e0d-4d20-9fe4-d023498face4 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:11.928036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e4078-8159-4796-a31c-aefc1e30e495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:11.928679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e4078-8159-4796-a31c-aefc1e30e495 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:11.928963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e4078-8159-4796-a31c-aefc1e30e495 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:11.929248 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e4078-8159-4796-a31c-aefc1e30e495 None admin] PATH_INFO: `/v3/users/3d07c5ee849548f4ac182934a84b2236` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:11.931550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e4078-8159-4796-a31c-aefc1e30e495 None admin] RBAC: Authorizing `identity:get_user(user_id=3d07c5ee849548f4ac182934a84b2236)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:11.931941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e4078-8159-4796-a31c-aefc1e30e495 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:11.933626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 551/1103] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:11 2022] GET /identity/v3/users/3d07c5ee849548f4ac182934a84b2236 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:11.940166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e1dad-7727-4407-b080-d6691d408921 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:12.044731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e1dad-7727-4407-b080-d6691d408921 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:12.067283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:12.081392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1a51a-f044-4c4b-9f91-9e3914ce3595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:12.082112 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1a51a-f044-4c4b-9f91-9e3914ce3595 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:12.082397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1a51a-f044-4c4b-9f91-9e3914ce3595 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:12.082669 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1a51a-f044-4c4b-9f91-9e3914ce3595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:12.152824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1a51a-f044-4c4b-9f91-9e3914ce3595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:12.154280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1a51a-f044-4c4b-9f91-9e3914ce3595 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:12.262253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 553/1104] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 21:23:12.268846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e4078-8159-4796-a31c-aefc1e30e495 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:12.371824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e4078-8159-4796-a31c-aefc1e30e495 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:12.407603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff971a6d-853f-400d-ac84-0a2457ccdb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:12.408183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff971a6d-853f-400d-ac84-0a2457ccdb3a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:12.408462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff971a6d-853f-400d-ac84-0a2457ccdb3a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:12.408734 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff971a6d-853f-400d-ac84-0a2457ccdb3a None admin] PATH_INFO: `/v3/projects/11b97170740f493fa15e6bf7b0a27dd4` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:12.409898 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff971a6d-853f-400d-ac84-0a2457ccdb3a None admin] RBAC: Authorizing `identity:get_project(project_id=11b97170740f493fa15e6bf7b0a27dd4)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:12.411344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff971a6d-853f-400d-ac84-0a2457ccdb3a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:12.412988 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 552/1105] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:12 2022] GET /identity/v3/projects/11b97170740f493fa15e6bf7b0a27dd4 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:12.419451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1a51a-f044-4c4b-9f91-9e3914ce3595 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:12.442404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:12.525087 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1a51a-f044-4c4b-9f91-9e3914ce3595 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:12.561947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7e866-9d03-4c6a-87d4-ecbe07d1c122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:12.562668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7e866-9d03-4c6a-87d4-ecbe07d1c122 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:12.562951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7e866-9d03-4c6a-87d4-ecbe07d1c122 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:12.563318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7e866-9d03-4c6a-87d4-ecbe07d1c122 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:12.564198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7e866-9d03-4c6a-87d4-ecbe07d1c122 None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=11b97170740f493fa15e6bf7b0a27dd4, user.id=3d07c5ee849548f4ac182934a84b2236) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:12.564513 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7e866-9d03-4c6a-87d4-ecbe07d1c122 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:12.566084 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7e866-9d03-4c6a-87d4-ecbe07d1c122 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:12.572952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 554/1106] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:23:12 2022] GET /identity/v3/role_assignments?user.id=3d07c5ee849548f4ac182934a84b2236&scope.project.id=11b97170740f493fa15e6bf7b0a27dd4&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:14.646890 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44af37-94da-401b-949e-04dc508c4076 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:14.647450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44af37-94da-401b-949e-04dc508c4076 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:14.647543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44af37-94da-401b-949e-04dc508c4076 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:14.648913 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 553/1107] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:14.654276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939a1f3-e3d8-4ffa-8ed3-55a5d6b69f60 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:14.654728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939a1f3-e3d8-4ffa-8ed3-55a5d6b69f60 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:14.654836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939a1f3-e3d8-4ffa-8ed3-55a5d6b69f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:14.673059 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-b939a1f3-e3d8-4ffa-8ed3-55a5d6b69f60 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:14.674897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b939a1f3-e3d8-4ffa-8ed3-55a5d6b69f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:14.745604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 555/1108] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:14.771190 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44af37-94da-401b-949e-04dc508c4076 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:14.796005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:14.882497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44af37-94da-401b-949e-04dc508c4076 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:14.917527 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfe5b1-0466-4b4a-ba3d-1c44845490e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:14.918213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfe5b1-0466-4b4a-ba3d-1c44845490e1 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:14.918496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfe5b1-0466-4b4a-ba3d-1c44845490e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:14.918771 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfe5b1-0466-4b4a-ba3d-1c44845490e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:14.919935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfe5b1-0466-4b4a-ba3d-1c44845490e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:14.921348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfe5b1-0466-4b4a-ba3d-1c44845490e1 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:14.923542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 554/1109] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:14.929563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939a1f3-e3d8-4ffa-8ed3-55a5d6b69f60 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:14.945404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:15.038214 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939a1f3-e3d8-4ffa-8ed3-55a5d6b69f60 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:15.075709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d4fea-0803-4bc4-a61d-46333a3306f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:15.076244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d4fea-0803-4bc4-a61d-46333a3306f5 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:15.076523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d4fea-0803-4bc4-a61d-46333a3306f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:15.076792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d4fea-0803-4bc4-a61d-46333a3306f5 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:15.077492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d4fea-0803-4bc4-a61d-46333a3306f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:15.079549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d4fea-0803-4bc4-a61d-46333a3306f5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:15.191091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 556/1110] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:23:14 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:23:17.288924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8af54-bb2f-4f10-868a-14c955729d53 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:17.289572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8af54-bb2f-4f10-868a-14c955729d53 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:17.289572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8af54-bb2f-4f10-868a-14c955729d53 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:17.291834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 555/1111] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:17.296291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974052-94d1-4b04-876b-d0988e428fdd None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:17.296736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974052-94d1-4b04-876b-d0988e428fdd None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:17.296839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974052-94d1-4b04-876b-d0988e428fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:17.317072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-e2974052-94d1-4b04-876b-d0988e428fdd None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:17.318300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e2974052-94d1-4b04-876b-d0988e428fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:17.380921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:17.385451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 557/1112] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:23:17.408922 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8af54-bb2f-4f10-868a-14c955729d53 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:17.501512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:17.512791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8af54-bb2f-4f10-868a-14c955729d53 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:17.549586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364b409-ea5f-4cb5-827a-aa1c29299e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:17.550207 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364b409-ea5f-4cb5-827a-aa1c29299e1d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:17.550485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364b409-ea5f-4cb5-827a-aa1c29299e1d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:17.550751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364b409-ea5f-4cb5-827a-aa1c29299e1d None admin] PATH_INFO: `/v3/roles/677f038038444f229f95fc80c3166f24` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:17.552188 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364b409-ea5f-4cb5-827a-aa1c29299e1d None admin] RBAC: Authorizing `identity:get_role(role_id=677f038038444f229f95fc80c3166f24)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:17.553545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364b409-ea5f-4cb5-827a-aa1c29299e1d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:17.555042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 556/1113] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:17 2022] GET /identity/v3/roles/677f038038444f229f95fc80c3166f24 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:17.560848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2974052-94d1-4b04-876b-d0988e428fdd None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:17.663885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2974052-94d1-4b04-876b-d0988e428fdd None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:17.698526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf191e84-ae44-4bdc-9ded-bc2c06fb64dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:17.699169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf191e84-ae44-4bdc-9ded-bc2c06fb64dd None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:17.699450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf191e84-ae44-4bdc-9ded-bc2c06fb64dd None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:17.699722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf191e84-ae44-4bdc-9ded-bc2c06fb64dd None admin] PATH_INFO: `/v3/users/75fb727afd1a42ad829c71cab9954d87` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:17.715372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf191e84-ae44-4bdc-9ded-bc2c06fb64dd None admin] RBAC: Authorizing `identity:get_user(user_id=75fb727afd1a42ad829c71cab9954d87)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:17.716824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf191e84-ae44-4bdc-9ded-bc2c06fb64dd None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:17.718517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 558/1114] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:17 2022] GET /identity/v3/users/75fb727afd1a42ad829c71cab9954d87 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:17.724673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364b409-ea5f-4cb5-827a-aa1c29299e1d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:17.835315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364b409-ea5f-4cb5-827a-aa1c29299e1d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:17.870330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b32882-00cc-430d-93c9-5b6f649dd73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:17.870953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b32882-00cc-430d-93c9-5b6f649dd73a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:17.871237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b32882-00cc-430d-93c9-5b6f649dd73a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:17.871513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b32882-00cc-430d-93c9-5b6f649dd73a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:17.899731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:17.944514 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b32882-00cc-430d-93c9-5b6f649dd73a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:17.949328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b32882-00cc-430d-93c9-5b6f649dd73a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:18.050347 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 557/1115] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:23:18.056447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf191e84-ae44-4bdc-9ded-bc2c06fb64dd None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:18.117451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:18.165939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf191e84-ae44-4bdc-9ded-bc2c06fb64dd None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:18.200943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f10d9-dd0d-4f62-b741-335c66411bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:18.201608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f10d9-dd0d-4f62-b741-335c66411bdc None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:18.201892 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f10d9-dd0d-4f62-b741-335c66411bdc None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:18.202200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f10d9-dd0d-4f62-b741-335c66411bdc None admin] PATH_INFO: `/v3/projects/11b97170740f493fa15e6bf7b0a27dd4` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:18.203355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f10d9-dd0d-4f62-b741-335c66411bdc None admin] RBAC: Authorizing `identity:get_project(project_id=11b97170740f493fa15e6bf7b0a27dd4)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:18.204924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f10d9-dd0d-4f62-b741-335c66411bdc None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:18.209671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 559/1116] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:18 2022] GET /identity/v3/projects/11b97170740f493fa15e6bf7b0a27dd4 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:18.212822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b32882-00cc-430d-93c9-5b6f649dd73a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:18.273840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:18.315915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b32882-00cc-430d-93c9-5b6f649dd73a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:18.353459 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c09e6b-cdc2-437b-96da-c391c415158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:18.354104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c09e6b-cdc2-437b-96da-c391c415158a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:18.354384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c09e6b-cdc2-437b-96da-c391c415158a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:18.354655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c09e6b-cdc2-437b-96da-c391c415158a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:18.355443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c09e6b-cdc2-437b-96da-c391c415158a None admin] RBAC: Adding query filter params (role.id=677f038038444f229f95fc80c3166f24, scope.project.id=11b97170740f493fa15e6bf7b0a27dd4, user.id=75fb727afd1a42ad829c71cab9954d87) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:18.355752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c09e6b-cdc2-437b-96da-c391c415158a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:18.357182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c09e6b-cdc2-437b-96da-c391c415158a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:18.364067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 558/1117] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:23:18 2022] GET /identity/v3/role_assignments?user.id=75fb727afd1a42ad829c71cab9954d87&scope.project.id=11b97170740f493fa15e6bf7b0a27dd4&role.id=677f038038444f229f95fc80c3166f24 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:20.419617 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3eb40-46ba-4d18-94ca-4656e17867b7 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:20.420192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3eb40-46ba-4d18-94ca-4656e17867b7 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:20.420192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3eb40-46ba-4d18-94ca-4656e17867b7 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:20.421488 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 560/1118] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:20.427498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2432d-7dfa-4ac0-8859-369a752f7dc5 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:20.427979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2432d-7dfa-4ac0-8859-369a752f7dc5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:20.428073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2432d-7dfa-4ac0-8859-369a752f7dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:20.447098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-b2e2432d-7dfa-4ac0-8859-369a752f7dc5 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:20.448302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b2e2432d-7dfa-4ac0-8859-369a752f7dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:20.515166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 559/1119] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:20.538494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3eb40-46ba-4d18-94ca-4656e17867b7 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:20.651453 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3eb40-46ba-4d18-94ca-4656e17867b7 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:20.677495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:20.686144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd30acd-e522-4575-ac53-330416ee1f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:20.690312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd30acd-e522-4575-ac53-330416ee1f04 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:20.690312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd30acd-e522-4575-ac53-330416ee1f04 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:20.690312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd30acd-e522-4575-ac53-330416ee1f04 None admin] PATH_INFO: `/v3/roles/677f038038444f229f95fc80c3166f24` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:20.690312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd30acd-e522-4575-ac53-330416ee1f04 None admin] RBAC: Authorizing `identity:get_role(role_id=677f038038444f229f95fc80c3166f24)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:20.691054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd30acd-e522-4575-ac53-330416ee1f04 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:20.692441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 561/1120] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:20 2022] GET /identity/v3/roles/677f038038444f229f95fc80c3166f24 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:20.698572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2432d-7dfa-4ac0-8859-369a752f7dc5 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:20.753137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:20.803092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2432d-7dfa-4ac0-8859-369a752f7dc5 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:20.839434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d606a-aab1-49de-8c06-e86f471f74b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:20.840080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d606a-aab1-49de-8c06-e86f471f74b3 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:20.840360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d606a-aab1-49de-8c06-e86f471f74b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:20.840635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d606a-aab1-49de-8c06-e86f471f74b3 None admin] PATH_INFO: `/v3/users/75fb727afd1a42ad829c71cab9954d87` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:20.841795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d606a-aab1-49de-8c06-e86f471f74b3 None admin] RBAC: Authorizing `identity:get_user(user_id=75fb727afd1a42ad829c71cab9954d87)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:20.843249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d606a-aab1-49de-8c06-e86f471f74b3 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:20.845141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 560/1121] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:20 2022] GET /identity/v3/users/75fb727afd1a42ad829c71cab9954d87 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:20.851372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd30acd-e522-4575-ac53-330416ee1f04 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:20.955754 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd30acd-e522-4575-ac53-330416ee1f04 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:20.990891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe8a57-a4b2-4a51-9fdc-1a9352e6b0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:20.991526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe8a57-a4b2-4a51-9fdc-1a9352e6b0ee None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:20.991808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe8a57-a4b2-4a51-9fdc-1a9352e6b0ee None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:20.992083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe8a57-a4b2-4a51-9fdc-1a9352e6b0ee None admin] PATH_INFO: `/v3/projects/11b97170740f493fa15e6bf7b0a27dd4` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:20.993279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe8a57-a4b2-4a51-9fdc-1a9352e6b0ee None admin] RBAC: Authorizing `identity:get_project(project_id=11b97170740f493fa15e6bf7b0a27dd4)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:20.994742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe8a57-a4b2-4a51-9fdc-1a9352e6b0ee None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:20.996402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 562/1122] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:20 2022] GET /identity/v3/projects/11b97170740f493fa15e6bf7b0a27dd4 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:21.003661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d606a-aab1-49de-8c06-e86f471f74b3 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:21.113851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d606a-aab1-49de-8c06-e86f471f74b3 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:21.138557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:21.153132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfd4e8-9bae-4bf5-bb88-ba7017d2576d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:21.153810 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfd4e8-9bae-4bf5-bb88-ba7017d2576d None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:21.154889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfd4e8-9bae-4bf5-bb88-ba7017d2576d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:21.154889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfd4e8-9bae-4bf5-bb88-ba7017d2576d None admin] PATH_INFO: `/v3/projects/11b97170740f493fa15e6bf7b0a27dd4/users/75fb727afd1a42ad829c71cab9954d87/roles/677f038038444f229f95fc80c3166f24` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:21.156793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfd4e8-9bae-4bf5-bb88-ba7017d2576d None admin] RBAC: Authorizing `identity:create_grant(project_id=11b97170740f493fa15e6bf7b0a27dd4, user_id=75fb727afd1a42ad829c71cab9954d87, role_id=677f038038444f229f95fc80c3166f24)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:21.158254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfd4e8-9bae-4bf5-bb88-ba7017d2576d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:21.231699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 561/1123] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:23:21 2022] PUT /identity/v3/projects/11b97170740f493fa15e6bf7b0a27dd4/users/75fb727afd1a42ad829c71cab9954d87/roles/677f038038444f229f95fc80c3166f24 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:23:23.262146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d3ffa-f62a-4fcb-91ab-b178f69dae3f None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:23.262716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d3ffa-f62a-4fcb-91ab-b178f69dae3f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:23.262716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d3ffa-f62a-4fcb-91ab-b178f69dae3f None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:23.264006 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 563/1124] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:23.270749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f2bcc-5b31-4923-bd73-143c831123bb None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:23.271242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f2bcc-5b31-4923-bd73-143c831123bb None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:23.271602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f2bcc-5b31-4923-bd73-143c831123bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:23.296322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1a0f2bcc-5b31-4923-bd73-143c831123bb None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:23.298278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1a0f2bcc-5b31-4923-bd73-143c831123bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:23.370916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 562/1125] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:23.394327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d3ffa-f62a-4fcb-91ab-b178f69dae3f None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:23.464627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:23.503167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d3ffa-f62a-4fcb-91ab-b178f69dae3f None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:23.539145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170aabc2-6465-4760-9f5a-18a4e4169acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:23.539810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170aabc2-6465-4760-9f5a-18a4e4169acd None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:23.540094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170aabc2-6465-4760-9f5a-18a4e4169acd None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:23.540370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170aabc2-6465-4760-9f5a-18a4e4169acd None admin] PATH_INFO: `/v3/roles/677f038038444f229f95fc80c3166f24` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:23.541818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170aabc2-6465-4760-9f5a-18a4e4169acd None admin] RBAC: Authorizing `identity:get_role(role_id=677f038038444f229f95fc80c3166f24)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:23.543291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170aabc2-6465-4760-9f5a-18a4e4169acd None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:23.544629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 564/1126] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:23 2022] GET /identity/v3/roles/677f038038444f229f95fc80c3166f24 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:23.550848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f2bcc-5b31-4923-bd73-143c831123bb None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:23.649913 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f2bcc-5b31-4923-bd73-143c831123bb None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:23.661850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:23.685199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c580000-e630-4f0e-894a-344f7115cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:23.685838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c580000-e630-4f0e-894a-344f7115cb29 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:23.686153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c580000-e630-4f0e-894a-344f7115cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:23.686429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c580000-e630-4f0e-894a-344f7115cb29 None admin] PATH_INFO: `/v3/users/75fb727afd1a42ad829c71cab9954d87` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:23.687591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c580000-e630-4f0e-894a-344f7115cb29 None admin] RBAC: Authorizing `identity:get_user(user_id=75fb727afd1a42ad829c71cab9954d87)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:23.689818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c580000-e630-4f0e-894a-344f7115cb29 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:23.691492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 563/1127] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:23 2022] GET /identity/v3/users/75fb727afd1a42ad829c71cab9954d87 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:23.697318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170aabc2-6465-4760-9f5a-18a4e4169acd None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:23.807135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170aabc2-6465-4760-9f5a-18a4e4169acd None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:23.837462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:23.843477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a96cd-f004-47f6-9e49-bc47dbf36e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:23.844643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a96cd-f004-47f6-9e49-bc47dbf36e9b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:23.845015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a96cd-f004-47f6-9e49-bc47dbf36e9b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:23.845310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a96cd-f004-47f6-9e49-bc47dbf36e9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:23.917958 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a96cd-f004-47f6-9e49-bc47dbf36e9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:23.919458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a96cd-f004-47f6-9e49-bc47dbf36e9b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:24.020103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 565/1128] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:23:24.026034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c580000-e630-4f0e-894a-344f7115cb29 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:24.140245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c580000-e630-4f0e-894a-344f7115cb29 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:24.178073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ba7bb-40e6-4ccb-965d-246928fbe9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:24.178888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ba7bb-40e6-4ccb-965d-246928fbe9a9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:24.179176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ba7bb-40e6-4ccb-965d-246928fbe9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:24.179457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ba7bb-40e6-4ccb-965d-246928fbe9a9 None admin] PATH_INFO: `/v3/projects/11b97170740f493fa15e6bf7b0a27dd4` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:24.180639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ba7bb-40e6-4ccb-965d-246928fbe9a9 None admin] RBAC: Authorizing `identity:get_project(project_id=11b97170740f493fa15e6bf7b0a27dd4)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:24.182098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ba7bb-40e6-4ccb-965d-246928fbe9a9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:24.183844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 564/1129] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:24 2022] GET /identity/v3/projects/11b97170740f493fa15e6bf7b0a27dd4 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:24.190058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a96cd-f004-47f6-9e49-bc47dbf36e9b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:24.220930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:24.299960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a96cd-f004-47f6-9e49-bc47dbf36e9b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:24.338493 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b5b67-3e2f-42ff-b94a-cb594adad7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:24.339121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b5b67-3e2f-42ff-b94a-cb594adad7cc None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:24.339404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b5b67-3e2f-42ff-b94a-cb594adad7cc None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:24.339675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b5b67-3e2f-42ff-b94a-cb594adad7cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:24.340468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b5b67-3e2f-42ff-b94a-cb594adad7cc None admin] RBAC: Adding query filter params (role.id=677f038038444f229f95fc80c3166f24, scope.project.id=11b97170740f493fa15e6bf7b0a27dd4, user.id=75fb727afd1a42ad829c71cab9954d87) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:24.340775 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b5b67-3e2f-42ff-b94a-cb594adad7cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:24.342232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b5b67-3e2f-42ff-b94a-cb594adad7cc None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:24.348365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 566/1130] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:23:24 2022] GET /identity/v3/role_assignments?user.id=75fb727afd1a42ad829c71cab9954d87&scope.project.id=11b97170740f493fa15e6bf7b0a27dd4&role.id=677f038038444f229f95fc80c3166f24 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:26.421482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217023fd-6a43-4692-bcdf-6c9e0a027c2b None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:26.422066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217023fd-6a43-4692-bcdf-6c9e0a027c2b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:26.422066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217023fd-6a43-4692-bcdf-6c9e0a027c2b None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:26.423426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 565/1131] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:23:26.431800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac69e7-7fca-4115-8c8f-d40c8f698eba None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:26.432333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac69e7-7fca-4115-8c8f-d40c8f698eba None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:26.432333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac69e7-7fca-4115-8c8f-d40c8f698eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:26.450669 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-daac69e7-7fca-4115-8c8f-d40c8f698eba None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:26.451851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-daac69e7-7fca-4115-8c8f-d40c8f698eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:26.528366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 567/1132] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:23:26.552402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217023fd-6a43-4692-bcdf-6c9e0a027c2b None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:26.600611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:26.658729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217023fd-6a43-4692-bcdf-6c9e0a027c2b None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:26.694755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fb9b9-3fd0-41a0-a355-3ac5c49cfc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:26.695386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fb9b9-3fd0-41a0-a355-3ac5c49cfc61 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:26.695662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fb9b9-3fd0-41a0-a355-3ac5c49cfc61 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:26.695930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fb9b9-3fd0-41a0-a355-3ac5c49cfc61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:26.697074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fb9b9-3fd0-41a0-a355-3ac5c49cfc61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:26.698519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fb9b9-3fd0-41a0-a355-3ac5c49cfc61 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:26.700096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 566/1133] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:26.706357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac69e7-7fca-4115-8c8f-d40c8f698eba None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:26.730891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:26.808209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac69e7-7fca-4115-8c8f-d40c8f698eba None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:26.849180 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00535377-ae36-4a8f-bc06-788f240088c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:26.849800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535377-ae36-4a8f-bc06-788f240088c2 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:26.850108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535377-ae36-4a8f-bc06-788f240088c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:26.850396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535377-ae36-4a8f-bc06-788f240088c2 None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:26.851088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00535377-ae36-4a8f-bc06-788f240088c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:26.852515 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00535377-ae36-4a8f-bc06-788f240088c2 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:26.964807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 568/1134] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:23:26 2022] POST /identity/v3/projects => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:23:29.050425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe6a11-5cb4-4ccd-ad71-6fe51441ebe8 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:29.051143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe6a11-5cb4-4ccd-ad71-6fe51441ebe8 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:29.051143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe6a11-5cb4-4ccd-ad71-6fe51441ebe8 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:29.052339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 567/1135] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:29.057897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232a99b-63ee-42e9-b92a-7aa0ca724b85 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:29.058424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232a99b-63ee-42e9-b92a-7aa0ca724b85 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:29.058542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232a99b-63ee-42e9-b92a-7aa0ca724b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:29.080199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c232a99b-63ee-42e9-b92a-7aa0ca724b85 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:29.081502 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c232a99b-63ee-42e9-b92a-7aa0ca724b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:29.156857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 569/1136] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:29.183506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe6a11-5cb4-4ccd-ad71-6fe51441ebe8 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:29.295561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe6a11-5cb4-4ccd-ad71-6fe51441ebe8 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:29.311321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:29.332963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696167a-6842-4edf-97ef-97b901b16a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:29.333648 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696167a-6842-4edf-97ef-97b901b16a1e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:29.333900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696167a-6842-4edf-97ef-97b901b16a1e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:29.334208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696167a-6842-4edf-97ef-97b901b16a1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:29.335404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696167a-6842-4edf-97ef-97b901b16a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:29.336837 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696167a-6842-4edf-97ef-97b901b16a1e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:29.338508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 568/1137] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:23:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:29.345649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232a99b-63ee-42e9-b92a-7aa0ca724b85 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:29.354144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:29.461601 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232a99b-63ee-42e9-b92a-7aa0ca724b85 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:29.503070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b925b-591e-4f35-8f2c-75718460d0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:29.503704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b925b-591e-4f35-8f2c-75718460d0c9 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:29.503987 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b925b-591e-4f35-8f2c-75718460d0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:29.504260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b925b-591e-4f35-8f2c-75718460d0c9 None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:29.504973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b925b-591e-4f35-8f2c-75718460d0c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:29.506469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b925b-591e-4f35-8f2c-75718460d0c9 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:29.633027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 570/1138] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:23:29 2022] POST /identity/v3/users => generated 286 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:23:31.786720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87f24f-796e-4721-a189-310d58799e82 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:31.787223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87f24f-796e-4721-a189-310d58799e82 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:31.787223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87f24f-796e-4721-a189-310d58799e82 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:31.788519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 569/1139] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:31.793731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f4cd9-4605-4548-9a12-07f7d1571471 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:31.794143 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f4cd9-4605-4548-9a12-07f7d1571471 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:31.794308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f4cd9-4605-4548-9a12-07f7d1571471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:31.812334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-9f1f4cd9-4605-4548-9a12-07f7d1571471 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:31.813532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9f1f4cd9-4605-4548-9a12-07f7d1571471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:31.870881 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:31.881006 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 571/1140] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:31.904877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87f24f-796e-4721-a189-310d58799e82 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:32.005043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87f24f-796e-4721-a189-310d58799e82 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:32.039421 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e854154-c25e-4726-a0b0-e216507e3795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:32.039977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e854154-c25e-4726-a0b0-e216507e3795 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:32.040258 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e854154-c25e-4726-a0b0-e216507e3795 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:32.040530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e854154-c25e-4726-a0b0-e216507e3795 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:32.050548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e854154-c25e-4726-a0b0-e216507e3795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-5e854154-c25e-4726-a0b0-e216507e3795 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:23:32.055530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:32.056963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:32.058152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 570/1141] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:23:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:32.062797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f4cd9-4605-4548-9a12-07f7d1571471 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:32.157865 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f4cd9-4605-4548-9a12-07f7d1571471 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:32.190526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baf470-b0fa-4c9e-b857-f022b9f787b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:32.191134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baf470-b0fa-4c9e-b857-f022b9f787b1 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:32.191409 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baf470-b0fa-4c9e-b857-f022b9f787b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:32.191678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baf470-b0fa-4c9e-b857-f022b9f787b1 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:32.192382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4baf470-b0fa-4c9e-b857-f022b9f787b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:32.192685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4baf470-b0fa-4c9e-b857-f022b9f787b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:32.194061 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4baf470-b0fa-4c9e-b857-f022b9f787b1 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:32.201837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 572/1142] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:23:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:32.207811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e854154-c25e-4726-a0b0-e216507e3795 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:32.253428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:32.307615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e854154-c25e-4726-a0b0-e216507e3795 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:32.340166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c1c90-d96e-4fcb-afce-bce21790560d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:32.340797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c1c90-d96e-4fcb-afce-bce21790560d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:32.341081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c1c90-d96e-4fcb-afce-bce21790560d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:32.341356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c1c90-d96e-4fcb-afce-bce21790560d None admin] PATH_INFO: `/v3/users/44f8da2bf3474eadab3a370485640d59` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:32.356800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c1c90-d96e-4fcb-afce-bce21790560d None admin] RBAC: Authorizing `identity:get_user(user_id=44f8da2bf3474eadab3a370485640d59)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:32.358278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c1c90-d96e-4fcb-afce-bce21790560d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:32.360618 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 571/1143] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:32 2022] GET /identity/v3/users/44f8da2bf3474eadab3a370485640d59 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:32.366908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baf470-b0fa-4c9e-b857-f022b9f787b1 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:32.420649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:32.465200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baf470-b0fa-4c9e-b857-f022b9f787b1 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:32.498301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed15609-1ba4-4c43-b503-3984b9678ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:32.498905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed15609-1ba4-4c43-b503-3984b9678ac5 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:32.499184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed15609-1ba4-4c43-b503-3984b9678ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:32.499454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed15609-1ba4-4c43-b503-3984b9678ac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:32.567572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed15609-1ba4-4c43-b503-3984b9678ac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:32.568949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed15609-1ba4-4c43-b503-3984b9678ac5 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:32.621341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:32.665166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 573/1144] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:23:32.671164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c1c90-d96e-4fcb-afce-bce21790560d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:32.764240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:32.767625 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c1c90-d96e-4fcb-afce-bce21790560d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:32.804211 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18ecd8-95ce-42cb-9dbc-fea9f6ce7823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:32.804908 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18ecd8-95ce-42cb-9dbc-fea9f6ce7823 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:32.805093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18ecd8-95ce-42cb-9dbc-fea9f6ce7823 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:32.805362 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18ecd8-95ce-42cb-9dbc-fea9f6ce7823 None admin] PATH_INFO: `/v3/projects/3f21e07a8f1649a3a18e21393effe08d` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:32.806528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18ecd8-95ce-42cb-9dbc-fea9f6ce7823 None admin] RBAC: Authorizing `identity:get_project(project_id=3f21e07a8f1649a3a18e21393effe08d)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:32.807949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18ecd8-95ce-42cb-9dbc-fea9f6ce7823 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:32.809585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 572/1145] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:32 2022] GET /identity/v3/projects/3f21e07a8f1649a3a18e21393effe08d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:32.815816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed15609-1ba4-4c43-b503-3984b9678ac5 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:32.925695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed15609-1ba4-4c43-b503-3984b9678ac5 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:32.960326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b81dd4-e9f4-406d-af6e-d57c4a55d6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:32.960941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b81dd4-e9f4-406d-af6e-d57c4a55d6a3 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:32.961217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b81dd4-e9f4-406d-af6e-d57c4a55d6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:32.961490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b81dd4-e9f4-406d-af6e-d57c4a55d6a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:32.962294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b81dd4-e9f4-406d-af6e-d57c4a55d6a3 None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=3f21e07a8f1649a3a18e21393effe08d, user.id=44f8da2bf3474eadab3a370485640d59) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:32.962605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b81dd4-e9f4-406d-af6e-d57c4a55d6a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:32.964024 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b81dd4-e9f4-406d-af6e-d57c4a55d6a3 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:32.969899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 574/1146] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:23:32 2022] GET /identity/v3/role_assignments?user.id=44f8da2bf3474eadab3a370485640d59&scope.project.id=3f21e07a8f1649a3a18e21393effe08d&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:34.976286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe3e6-43d3-401b-882d-294bc0e10a4e None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:34.976844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe3e6-43d3-401b-882d-294bc0e10a4e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:34.976844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe3e6-43d3-401b-882d-294bc0e10a4e None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:34.978206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 573/1147] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:34.983479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30988-be66-4847-8de7-a04d15d50ad2 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:34.983887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30988-be66-4847-8de7-a04d15d50ad2 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:34.984032 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30988-be66-4847-8de7-a04d15d50ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:35.003279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-fcd30988-be66-4847-8de7-a04d15d50ad2 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:35.004765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-fcd30988-be66-4847-8de7-a04d15d50ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:35.019450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:35.076926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 575/1148] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:35.099911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfe3e6-43d3-401b-882d-294bc0e10a4e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:35.202960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfe3e6-43d3-401b-882d-294bc0e10a4e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:35.238266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ac597-4351-4b77-8285-b6306e022f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:35.238806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ac597-4351-4b77-8285-b6306e022f1f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:35.239110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ac597-4351-4b77-8285-b6306e022f1f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:35.239380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ac597-4351-4b77-8285-b6306e022f1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:35.250656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ac597-4351-4b77-8285-b6306e022f1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-9d9ac597-4351-4b77-8285-b6306e022f1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:23:35.255430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:35.256533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:35.257795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 574/1149] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:23:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:35.262995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd30988-be66-4847-8de7-a04d15d50ad2 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:35.364031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd30988-be66-4847-8de7-a04d15d50ad2 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:35.399520 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81223fc9-51b6-49c3-ad1c-40ec36b8f8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:35.400135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81223fc9-51b6-49c3-ad1c-40ec36b8f8c0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:35.400417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81223fc9-51b6-49c3-ad1c-40ec36b8f8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:35.400689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81223fc9-51b6-49c3-ad1c-40ec36b8f8c0 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:35.401438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81223fc9-51b6-49c3-ad1c-40ec36b8f8c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:35.401740 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81223fc9-51b6-49c3-ad1c-40ec36b8f8c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:35.403262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81223fc9-51b6-49c3-ad1c-40ec36b8f8c0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:35.411928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 576/1150] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:23:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:35.417867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ac597-4351-4b77-8285-b6306e022f1f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:35.447281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:35.525813 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ac597-4351-4b77-8285-b6306e022f1f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:35.560918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295cd4f-946a-4119-adbf-b602eee76abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:35.561579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295cd4f-946a-4119-adbf-b602eee76abb None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:35.561880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295cd4f-946a-4119-adbf-b602eee76abb None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:35.562194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295cd4f-946a-4119-adbf-b602eee76abb None admin] PATH_INFO: `/v3/users/44f8da2bf3474eadab3a370485640d59` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:35.564030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295cd4f-946a-4119-adbf-b602eee76abb None admin] RBAC: Authorizing `identity:get_user(user_id=44f8da2bf3474eadab3a370485640d59)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:35.565456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295cd4f-946a-4119-adbf-b602eee76abb None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:35.567103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 575/1151] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:35 2022] GET /identity/v3/users/44f8da2bf3474eadab3a370485640d59 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:35.573403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81223fc9-51b6-49c3-ad1c-40ec36b8f8c0 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:35.587602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:35.679256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81223fc9-51b6-49c3-ad1c-40ec36b8f8c0 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:35.714825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cddc49-6a1f-4028-ad3c-c50728544a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:35.715492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cddc49-6a1f-4028-ad3c-c50728544a54 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:35.715780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cddc49-6a1f-4028-ad3c-c50728544a54 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:35.716071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cddc49-6a1f-4028-ad3c-c50728544a54 None admin] PATH_INFO: `/v3/projects/3f21e07a8f1649a3a18e21393effe08d` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:35.717295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cddc49-6a1f-4028-ad3c-c50728544a54 None admin] RBAC: Authorizing `identity:get_project(project_id=3f21e07a8f1649a3a18e21393effe08d)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:35.718879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cddc49-6a1f-4028-ad3c-c50728544a54 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:35.720672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 577/1152] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:35 2022] GET /identity/v3/projects/3f21e07a8f1649a3a18e21393effe08d => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:35.728855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295cd4f-946a-4119-adbf-b602eee76abb None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:35.828001 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:35.834051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295cd4f-946a-4119-adbf-b602eee76abb None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:35.870143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab865d41-4cc6-4ca0-921e-ad1f34cff9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:35.870863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab865d41-4cc6-4ca0-921e-ad1f34cff9ee None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:35.871478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab865d41-4cc6-4ca0-921e-ad1f34cff9ee None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:35.871762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab865d41-4cc6-4ca0-921e-ad1f34cff9ee None admin] PATH_INFO: `/v3/projects/3f21e07a8f1649a3a18e21393effe08d/users/44f8da2bf3474eadab3a370485640d59/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:35.873702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab865d41-4cc6-4ca0-921e-ad1f34cff9ee None admin] RBAC: Authorizing `identity:create_grant(project_id=3f21e07a8f1649a3a18e21393effe08d, user_id=44f8da2bf3474eadab3a370485640d59, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:35.875402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab865d41-4cc6-4ca0-921e-ad1f34cff9ee None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:35.996912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 576/1153] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:23:35 2022] PUT /identity/v3/projects/3f21e07a8f1649a3a18e21393effe08d/users/44f8da2bf3474eadab3a370485640d59/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:23:38.054633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72e781-b5b8-4ed5-9d79-b177593c7ac6 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:38.055414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72e781-b5b8-4ed5-9d79-b177593c7ac6 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:38.055414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72e781-b5b8-4ed5-9d79-b177593c7ac6 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:38.056508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 578/1154] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:38.061669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0073e-e2f2-4013-b6a5-b8b747fea4b5 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:38.062122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0073e-e2f2-4013-b6a5-b8b747fea4b5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:38.062282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0073e-e2f2-4013-b6a5-b8b747fea4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:38.080901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-20d0073e-e2f2-4013-b6a5-b8b747fea4b5 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:38.082156 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-20d0073e-e2f2-4013-b6a5-b8b747fea4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:38.149649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 577/1155] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:38.173281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72e781-b5b8-4ed5-9d79-b177593c7ac6 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:38.254936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:38.279568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72e781-b5b8-4ed5-9d79-b177593c7ac6 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:38.316953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d9ace-3f7f-4a5e-95d0-bc2a6ef456bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:38.317533 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9ace-3f7f-4a5e-95d0-bc2a6ef456bc None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:38.317815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9ace-3f7f-4a5e-95d0-bc2a6ef456bc None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:38.318113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9ace-3f7f-4a5e-95d0-bc2a6ef456bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:38.328482 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d9ace-3f7f-4a5e-95d0-bc2a6ef456bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-db5d9ace-3f7f-4a5e-95d0-bc2a6ef456bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:23:38.334023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:38.335227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:23:38.336543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 579/1156] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:23:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:38.342189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0073e-e2f2-4013-b6a5-b8b747fea4b5 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:38.428357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:38.443440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0073e-e2f2-4013-b6a5-b8b747fea4b5 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:38.477091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55591bdf-e47f-44b8-b75c-cc92f612f673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:38.477700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55591bdf-e47f-44b8-b75c-cc92f612f673 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:38.477983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55591bdf-e47f-44b8-b75c-cc92f612f673 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:38.483610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55591bdf-e47f-44b8-b75c-cc92f612f673 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:38.483610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55591bdf-e47f-44b8-b75c-cc92f612f673 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:38.483610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55591bdf-e47f-44b8-b75c-cc92f612f673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:38.483610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55591bdf-e47f-44b8-b75c-cc92f612f673 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:38.489448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 578/1157] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:38.495416 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d9ace-3f7f-4a5e-95d0-bc2a6ef456bc None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:38.598405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d9ace-3f7f-4a5e-95d0-bc2a6ef456bc None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:38.633798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8061a56-833c-4c92-a9f9-b9fc3adb1eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:38.634466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8061a56-833c-4c92-a9f9-b9fc3adb1eb0 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:38.634750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8061a56-833c-4c92-a9f9-b9fc3adb1eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:38.635025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8061a56-833c-4c92-a9f9-b9fc3adb1eb0 None admin] PATH_INFO: `/v3/users/44f8da2bf3474eadab3a370485640d59` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:38.636194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8061a56-833c-4c92-a9f9-b9fc3adb1eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=44f8da2bf3474eadab3a370485640d59)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:38.637637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8061a56-833c-4c92-a9f9-b9fc3adb1eb0 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:38.639279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 580/1158] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:38 2022] GET /identity/v3/users/44f8da2bf3474eadab3a370485640d59 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:38.645204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55591bdf-e47f-44b8-b75c-cc92f612f673 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:38.752380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55591bdf-e47f-44b8-b75c-cc92f612f673 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:38.785413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669ec3b-6a22-40a2-945e-fd1295ddba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:38.786050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669ec3b-6a22-40a2-945e-fd1295ddba5d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:38.786336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669ec3b-6a22-40a2-945e-fd1295ddba5d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:38.786612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669ec3b-6a22-40a2-945e-fd1295ddba5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:38.805463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:38.857392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669ec3b-6a22-40a2-945e-fd1295ddba5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:38.858781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669ec3b-6a22-40a2-945e-fd1295ddba5d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:38.958993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 579/1159] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:23:38.965066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8061a56-833c-4c92-a9f9-b9fc3adb1eb0 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:38.974857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:39.067581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8061a56-833c-4c92-a9f9-b9fc3adb1eb0 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:39.106556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de7e83-4aa4-4c61-9e49-0de3fa36ab02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:39.107195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de7e83-4aa4-4c61-9e49-0de3fa36ab02 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:39.107532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de7e83-4aa4-4c61-9e49-0de3fa36ab02 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:39.107761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de7e83-4aa4-4c61-9e49-0de3fa36ab02 None admin] PATH_INFO: `/v3/projects/3f21e07a8f1649a3a18e21393effe08d` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:39.108942 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de7e83-4aa4-4c61-9e49-0de3fa36ab02 None admin] RBAC: Authorizing `identity:get_project(project_id=3f21e07a8f1649a3a18e21393effe08d)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:39.111400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de7e83-4aa4-4c61-9e49-0de3fa36ab02 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:39.113377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 581/1160] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:38 2022] GET /identity/v3/projects/3f21e07a8f1649a3a18e21393effe08d => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:39.119713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669ec3b-6a22-40a2-945e-fd1295ddba5d None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:39.171720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:39.225183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669ec3b-6a22-40a2-945e-fd1295ddba5d None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:39.268820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7989f-36fc-43ef-b3fa-81ec98356b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:39.269450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7989f-36fc-43ef-b3fa-81ec98356b0a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:39.269736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7989f-36fc-43ef-b3fa-81ec98356b0a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:39.270678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7989f-36fc-43ef-b3fa-81ec98356b0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:39.271490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7989f-36fc-43ef-b3fa-81ec98356b0a None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=3f21e07a8f1649a3a18e21393effe08d, user.id=44f8da2bf3474eadab3a370485640d59) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:39.271807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7989f-36fc-43ef-b3fa-81ec98356b0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:39.273242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7989f-36fc-43ef-b3fa-81ec98356b0a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:39.279575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 580/1161] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:23:39 2022] GET /identity/v3/role_assignments?user.id=44f8da2bf3474eadab3a370485640d59&scope.project.id=3f21e07a8f1649a3a18e21393effe08d&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:41.374841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406aa3a-b47c-47d9-99fc-604e86dac3a4 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:41.375447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406aa3a-b47c-47d9-99fc-604e86dac3a4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:41.375552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406aa3a-b47c-47d9-99fc-604e86dac3a4 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:41.376825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 582/1162] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:41.382265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06215a2b-c243-47ed-b9cc-1a7dc4174e6d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:41.382699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06215a2b-c243-47ed-b9cc-1a7dc4174e6d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:41.382833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06215a2b-c243-47ed-b9cc-1a7dc4174e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:41.400911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-06215a2b-c243-47ed-b9cc-1a7dc4174e6d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:41.402136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-06215a2b-c243-47ed-b9cc-1a7dc4174e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:41.468229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 581/1163] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:41.491067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406aa3a-b47c-47d9-99fc-604e86dac3a4 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:41.568007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:41.595702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406aa3a-b47c-47d9-99fc-604e86dac3a4 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:41.629899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7c2fe-d16a-4d79-994a-ae2264b1101d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:41.630453 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7c2fe-d16a-4d79-994a-ae2264b1101d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:41.630733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7c2fe-d16a-4d79-994a-ae2264b1101d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:41.631003 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7c2fe-d16a-4d79-994a-ae2264b1101d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:41.697735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7c2fe-d16a-4d79-994a-ae2264b1101d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:41.699277 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7c2fe-d16a-4d79-994a-ae2264b1101d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:41.774836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:41.802392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 583/1164] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:23:41.808821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06215a2b-c243-47ed-b9cc-1a7dc4174e6d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:41.853583 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:41.911983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06215a2b-c243-47ed-b9cc-1a7dc4174e6d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:41.946776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5a057-9f53-4ee9-aa99-564b73912a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:41.947424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5a057-9f53-4ee9-aa99-564b73912a92 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:41.947733 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5a057-9f53-4ee9-aa99-564b73912a92 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:41.948010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5a057-9f53-4ee9-aa99-564b73912a92 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:41.957991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5a057-9f53-4ee9-aa99-564b73912a92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:41.959460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5a057-9f53-4ee9-aa99-564b73912a92 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-d8d5a057-9f53-4ee9-aa99-564b73912a92 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:41.971399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:41.972784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:41.973826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 582/1165] 10.222.0.73 () {56 vars in 1117 bytes} [Thu Aug 4 21:23:41 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:41.978700 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7c2fe-d16a-4d79-994a-ae2264b1101d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:42.081085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7c2fe-d16a-4d79-994a-ae2264b1101d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:42.117923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318ea00-ed9a-40db-8adb-4376d97e32d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:42.118611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318ea00-ed9a-40db-8adb-4376d97e32d4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:42.118894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318ea00-ed9a-40db-8adb-4376d97e32d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:42.119166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318ea00-ed9a-40db-8adb-4376d97e32d4 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:42.119861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318ea00-ed9a-40db-8adb-4376d97e32d4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:42.120169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318ea00-ed9a-40db-8adb-4376d97e32d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:42.121550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318ea00-ed9a-40db-8adb-4376d97e32d4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:42.129169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 584/1166] 10.222.0.73 () {56 vars in 1115 bytes} [Thu Aug 4 21:23:41 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:44.151879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf592d10-28e8-4dde-97d8-cbabe854246f None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:44.152414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf592d10-28e8-4dde-97d8-cbabe854246f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:44.152414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf592d10-28e8-4dde-97d8-cbabe854246f None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:44.153758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 583/1167] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:44.159156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c960-e2f8-4f0a-9aa2-ae165f896db6 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:44.159572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c960-e2f8-4f0a-9aa2-ae165f896db6 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:44.159726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c960-e2f8-4f0a-9aa2-ae165f896db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:44.178113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1710c960-e2f8-4f0a-9aa2-ae165f896db6 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:44.179331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1710c960-e2f8-4f0a-9aa2-ae165f896db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:44.209549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:44.247496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 585/1168] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:44.271038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf592d10-28e8-4dde-97d8-cbabe854246f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:44.379531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf592d10-28e8-4dde-97d8-cbabe854246f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:44.381267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:44.415358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460b093-9ec5-4bf4-8199-c84a48d79e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:44.415892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460b093-9ec5-4bf4-8199-c84a48d79e6c None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:44.416182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460b093-9ec5-4bf4-8199-c84a48d79e6c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:44.416458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460b093-9ec5-4bf4-8199-c84a48d79e6c None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:44.417194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460b093-9ec5-4bf4-8199-c84a48d79e6c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:44.418614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460b093-9ec5-4bf4-8199-c84a48d79e6c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:44.529675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 584/1169] 10.222.0.73 () {60 vars in 1138 bytes} [Thu Aug 4 21:23:44 2022] POST /identity/v3/domains => generated 267 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:23:46.604559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86421d9-66ff-43e8-9163-3e53e178d30d None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:46.605079 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86421d9-66ff-43e8-9163-3e53e178d30d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:46.605079 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86421d9-66ff-43e8-9163-3e53e178d30d None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:46.606470 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 586/1170] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:46.612316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7d5f9-aed7-41ec-8e28-779e7ada549d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:46.612790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7d5f9-aed7-41ec-8e28-779e7ada549d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:46.612887 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7d5f9-aed7-41ec-8e28-779e7ada549d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:46.631447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-e3c7d5f9-aed7-41ec-8e28-779e7ada549d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:46.632648 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e3c7d5f9-aed7-41ec-8e28-779e7ada549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:46.702019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 585/1171] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:46.733330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86421d9-66ff-43e8-9163-3e53e178d30d None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:46.839213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86421d9-66ff-43e8-9163-3e53e178d30d None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:46.874209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d76dc8-ee39-4cf2-96bd-3e4110d4c26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:46.875407 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d76dc8-ee39-4cf2-96bd-3e4110d4c26c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:46.875689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d76dc8-ee39-4cf2-96bd-3e4110d4c26c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:46.875964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d76dc8-ee39-4cf2-96bd-3e4110d4c26c None admin] PATH_INFO: `/v3/domains/5ac2083a627c425c855c1854952ab53b` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:46.886971 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d76dc8-ee39-4cf2-96bd-3e4110d4c26c None admin] RBAC: Authorizing `identity:get_domain(domain_id=5ac2083a627c425c855c1854952ab53b)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:46.888438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d76dc8-ee39-4cf2-96bd-3e4110d4c26c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:46.890155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 587/1172] 10.222.0.73 () {56 vars in 1183 bytes} [Thu Aug 4 21:23:46 2022] GET /identity/v3/domains/5ac2083a627c425c855c1854952ab53b => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:46.896694 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7d5f9-aed7-41ec-8e28-779e7ada549d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:46.992513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:47.002814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7d5f9-aed7-41ec-8e28-779e7ada549d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:47.039810 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be87c36-4e80-4c05-a6f4-e93a58ce6e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:47.040581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be87c36-4e80-4c05-a6f4-e93a58ce6e3f None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:47.040927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be87c36-4e80-4c05-a6f4-e93a58ce6e3f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:47.041206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be87c36-4e80-4c05-a6f4-e93a58ce6e3f None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:47.041901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be87c36-4e80-4c05-a6f4-e93a58ce6e3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:47.043357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be87c36-4e80-4c05-a6f4-e93a58ce6e3f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:47.135553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 586/1173] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:23:46 2022] POST /identity/v3/projects => generated 361 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:23:49.420178 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aa2b0-752f-4e1a-9252-a00e4a8bc419 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:49.420782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aa2b0-752f-4e1a-9252-a00e4a8bc419 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:49.420782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aa2b0-752f-4e1a-9252-a00e4a8bc419 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:49.422287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 588/1174] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:49.428039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5c827-9f47-45d2-94af-e788cde374c1 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:49.428526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5c827-9f47-45d2-94af-e788cde374c1 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:49.428635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5c827-9f47-45d2-94af-e788cde374c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:49.447829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-91a5c827-9f47-45d2-94af-e788cde374c1 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:49.449029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-91a5c827-9f47-45d2-94af-e788cde374c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:49.520061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 587/1175] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:49.544381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576aa2b0-752f-4e1a-9252-a00e4a8bc419 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:49.574608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:49.650494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576aa2b0-752f-4e1a-9252-a00e4a8bc419 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:49.686111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a4cb1-70ed-49bd-ac81-37603523fb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:49.687266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a4cb1-70ed-49bd-ac81-37603523fb68 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:49.687685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a4cb1-70ed-49bd-ac81-37603523fb68 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:49.687966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a4cb1-70ed-49bd-ac81-37603523fb68 None admin] PATH_INFO: `/v3/domains/5ac2083a627c425c855c1854952ab53b` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:49.689134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a4cb1-70ed-49bd-ac81-37603523fb68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5ac2083a627c425c855c1854952ab53b)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:49.690600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a4cb1-70ed-49bd-ac81-37603523fb68 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:49.692183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 589/1176] 10.222.0.73 () {56 vars in 1183 bytes} [Thu Aug 4 21:23:49 2022] GET /identity/v3/domains/5ac2083a627c425c855c1854952ab53b => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:49.698091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5c827-9f47-45d2-94af-e788cde374c1 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:49.776320 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:49.801480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5c827-9f47-45d2-94af-e788cde374c1 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:49.835060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273913f6-535a-4a33-a2d0-3e889738df99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:49.835709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273913f6-535a-4a33-a2d0-3e889738df99 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:49.835986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273913f6-535a-4a33-a2d0-3e889738df99 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:49.836250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273913f6-535a-4a33-a2d0-3e889738df99 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:49.836932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273913f6-535a-4a33-a2d0-3e889738df99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:49.838449 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273913f6-535a-4a33-a2d0-3e889738df99 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:49.929438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 588/1177] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:23:49 2022] POST /identity/v3/users => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:23:52.084027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20600a25-ffb7-4770-8202-5ff18bc05353 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:52.084592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20600a25-ffb7-4770-8202-5ff18bc05353 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:52.084592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20600a25-ffb7-4770-8202-5ff18bc05353 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:52.085917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 590/1178] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:52.091527 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bc7ba-32a8-4715-aa00-0ce02f277fd4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:52.092002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bc7ba-32a8-4715-aa00-0ce02f277fd4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:52.092112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bc7ba-32a8-4715-aa00-0ce02f277fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:52.110870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-747bc7ba-32a8-4715-aa00-0ce02f277fd4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:52.112071 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-747bc7ba-32a8-4715-aa00-0ce02f277fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:52.185440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 589/1179] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:52.209737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20600a25-ffb7-4770-8202-5ff18bc05353 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:52.308335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:52.315819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20600a25-ffb7-4770-8202-5ff18bc05353 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:52.352017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a44bde-963f-4813-bb17-126caffb43a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:52.352850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a44bde-963f-4813-bb17-126caffb43a9 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:52.353300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a44bde-963f-4813-bb17-126caffb43a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:52.353596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a44bde-963f-4813-bb17-126caffb43a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:52.373980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a44bde-963f-4813-bb17-126caffb43a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-91a44bde-963f-4813-bb17-126caffb43a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:23:52.379043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:52.380231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:23:52.381146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 591/1180] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:23:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:52.386568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bc7ba-32a8-4715-aa00-0ce02f277fd4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:52.445089 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:52.487770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bc7ba-32a8-4715-aa00-0ce02f277fd4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:52.522758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c06a88-10f1-4beb-ab32-11fba9fb9dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:52.522962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c06a88-10f1-4beb-ab32-11fba9fb9dc2 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:52.523238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c06a88-10f1-4beb-ab32-11fba9fb9dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:52.523513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c06a88-10f1-4beb-ab32-11fba9fb9dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:52.524228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c06a88-10f1-4beb-ab32-11fba9fb9dc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:52.524529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c06a88-10f1-4beb-ab32-11fba9fb9dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:52.525882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c06a88-10f1-4beb-ab32-11fba9fb9dc2 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:52.535122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 590/1181] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:23:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:52.542385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a44bde-963f-4813-bb17-126caffb43a9 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:52.649622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a44bde-963f-4813-bb17-126caffb43a9 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:52.684277 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b09ec8-baa0-48d3-96f8-435485b0f347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:52.684985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b09ec8-baa0-48d3-96f8-435485b0f347 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:52.685262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b09ec8-baa0-48d3-96f8-435485b0f347 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:52.685538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b09ec8-baa0-48d3-96f8-435485b0f347 None admin] PATH_INFO: `/v3/users/aafd5a0da0c4423eabb33fcca0bbef7a` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:52.704614 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b09ec8-baa0-48d3-96f8-435485b0f347 None admin] RBAC: Authorizing `identity:get_user(user_id=aafd5a0da0c4423eabb33fcca0bbef7a)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:52.706245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b09ec8-baa0-48d3-96f8-435485b0f347 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:52.707939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 592/1182] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:52 2022] GET /identity/v3/users/aafd5a0da0c4423eabb33fcca0bbef7a => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:52.713859 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c06a88-10f1-4beb-ab32-11fba9fb9dc2 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:52.819928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c06a88-10f1-4beb-ab32-11fba9fb9dc2 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:52.843670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:52.859455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fae98-5645-44e1-a03d-f3f2896d0ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:52.860085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fae98-5645-44e1-a03d-f3f2896d0ce2 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:52.860371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fae98-5645-44e1-a03d-f3f2896d0ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:52.860646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fae98-5645-44e1-a03d-f3f2896d0ce2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:52.932568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fae98-5645-44e1-a03d-f3f2896d0ce2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:52.932785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fae98-5645-44e1-a03d-f3f2896d0ce2 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:53.037338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 591/1183] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:23:53.043247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b09ec8-baa0-48d3-96f8-435485b0f347 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:53.059444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:53.143298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b09ec8-baa0-48d3-96f8-435485b0f347 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:53.177571 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ed047-b8bc-41a5-9968-4b863898d8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:53.178241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ed047-b8bc-41a5-9968-4b863898d8a1 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:53.178522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ed047-b8bc-41a5-9968-4b863898d8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:53.178793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ed047-b8bc-41a5-9968-4b863898d8a1 None admin] PATH_INFO: `/v3/projects/efbde536d0a94747879bed317e8f1eda` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:53.179940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ed047-b8bc-41a5-9968-4b863898d8a1 None admin] RBAC: Authorizing `identity:get_project(project_id=efbde536d0a94747879bed317e8f1eda)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:53.181484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ed047-b8bc-41a5-9968-4b863898d8a1 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:53.183230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 593/1184] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:53 2022] GET /identity/v3/projects/efbde536d0a94747879bed317e8f1eda => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:53.189891 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fae98-5645-44e1-a03d-f3f2896d0ce2 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:53.214607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:53.293496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fae98-5645-44e1-a03d-f3f2896d0ce2 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:53.329458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8daad1d-5055-4739-bd77-85f08fb29546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:53.330095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daad1d-5055-4739-bd77-85f08fb29546 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:53.330383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daad1d-5055-4739-bd77-85f08fb29546 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:53.330658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daad1d-5055-4739-bd77-85f08fb29546 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:53.331442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8daad1d-5055-4739-bd77-85f08fb29546 None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=efbde536d0a94747879bed317e8f1eda, user.id=aafd5a0da0c4423eabb33fcca0bbef7a) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:53.331793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8daad1d-5055-4739-bd77-85f08fb29546 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:53.333246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8daad1d-5055-4739-bd77-85f08fb29546 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:53.339007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 592/1185] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:23:53 2022] GET /identity/v3/role_assignments?user.id=aafd5a0da0c4423eabb33fcca0bbef7a&scope.project.id=efbde536d0a94747879bed317e8f1eda&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:55.354468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217144d-4160-4627-88a1-347e9a95c70b None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:55.355045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217144d-4160-4627-88a1-347e9a95c70b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:55.355045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217144d-4160-4627-88a1-347e9a95c70b None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:55.356380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 594/1186] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:55.362572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ce15e-a37f-4365-bc5e-a3efa57ffe1a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:55.362982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ce15e-a37f-4365-bc5e-a3efa57ffe1a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:55.363121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ce15e-a37f-4365-bc5e-a3efa57ffe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:55.380849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c12ce15e-a37f-4365-bc5e-a3efa57ffe1a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:55.382043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c12ce15e-a37f-4365-bc5e-a3efa57ffe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:55.450799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 593/1187] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:55.473906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217144d-4160-4627-88a1-347e9a95c70b None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:55.569334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:55.589552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217144d-4160-4627-88a1-347e9a95c70b None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:55.626117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e52d-8ddd-4d8a-93ce-da6a4e0b5a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:55.626699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e52d-8ddd-4d8a-93ce-da6a4e0b5a46 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:55.626979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e52d-8ddd-4d8a-93ce-da6a4e0b5a46 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:55.627249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e52d-8ddd-4d8a-93ce-da6a4e0b5a46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:55.637440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3e52d-8ddd-4d8a-93ce-da6a4e0b5a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-3ae3e52d-8ddd-4d8a-93ce-da6a4e0b5a46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:23:55.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:55.644384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:23:55.645232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 595/1188] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:23:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:23:55.650379 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ce15e-a37f-4365-bc5e-a3efa57ffe1a None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:55.668052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:55.751204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ce15e-a37f-4365-bc5e-a3efa57ffe1a None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:55.784910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ca74b-6b62-484d-918e-330187f5843b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:55.785526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ca74b-6b62-484d-918e-330187f5843b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:55.785805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ca74b-6b62-484d-918e-330187f5843b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:55.786104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ca74b-6b62-484d-918e-330187f5843b None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:55.786812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ca74b-6b62-484d-918e-330187f5843b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:55.787111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ca74b-6b62-484d-918e-330187f5843b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:55.788468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ca74b-6b62-484d-918e-330187f5843b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:55.798124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 594/1189] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:23:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:55.804106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e52d-8ddd-4d8a-93ce-da6a4e0b5a46 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:55.903446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e52d-8ddd-4d8a-93ce-da6a4e0b5a46 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:55.937287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde959c-c912-41fb-8e45-4e7d1cc27f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:55.937929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde959c-c912-41fb-8e45-4e7d1cc27f64 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:55.938239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde959c-c912-41fb-8e45-4e7d1cc27f64 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:55.938516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde959c-c912-41fb-8e45-4e7d1cc27f64 None admin] PATH_INFO: `/v3/users/aafd5a0da0c4423eabb33fcca0bbef7a` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:55.939682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde959c-c912-41fb-8e45-4e7d1cc27f64 None admin] RBAC: Authorizing `identity:get_user(user_id=aafd5a0da0c4423eabb33fcca0bbef7a)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:55.941116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde959c-c912-41fb-8e45-4e7d1cc27f64 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:55.942743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 596/1190] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:55 2022] GET /identity/v3/users/aafd5a0da0c4423eabb33fcca0bbef7a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:55.948612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ca74b-6b62-484d-918e-330187f5843b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:56.027661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:56.050364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ca74b-6b62-484d-918e-330187f5843b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:56.083585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9186ba-251c-473e-9304-a6f311ba0216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:56.084219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9186ba-251c-473e-9304-a6f311ba0216 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:56.084499 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9186ba-251c-473e-9304-a6f311ba0216 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:56.084773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9186ba-251c-473e-9304-a6f311ba0216 None admin] PATH_INFO: `/v3/projects/efbde536d0a94747879bed317e8f1eda` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:56.085932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9186ba-251c-473e-9304-a6f311ba0216 None admin] RBAC: Authorizing `identity:get_project(project_id=efbde536d0a94747879bed317e8f1eda)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:56.087359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9186ba-251c-473e-9304-a6f311ba0216 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:56.089001 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 595/1191] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:55 2022] GET /identity/v3/projects/efbde536d0a94747879bed317e8f1eda => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:56.095507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde959c-c912-41fb-8e45-4e7d1cc27f64 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:56.108130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:56.201995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde959c-c912-41fb-8e45-4e7d1cc27f64 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:56.236204 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681625a8-4ae3-4cdb-846b-e36553f979cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:56.236864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681625a8-4ae3-4cdb-846b-e36553f979cc None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:56.237391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681625a8-4ae3-4cdb-846b-e36553f979cc None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:56.237671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681625a8-4ae3-4cdb-846b-e36553f979cc None admin] PATH_INFO: `/v3/projects/efbde536d0a94747879bed317e8f1eda/users/aafd5a0da0c4423eabb33fcca0bbef7a/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:56.239654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681625a8-4ae3-4cdb-846b-e36553f979cc None admin] RBAC: Authorizing `identity:create_grant(project_id=efbde536d0a94747879bed317e8f1eda, user_id=aafd5a0da0c4423eabb33fcca0bbef7a, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:56.241087 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681625a8-4ae3-4cdb-846b-e36553f979cc None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:56.322219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 597/1192] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:23:56 2022] PUT /identity/v3/projects/efbde536d0a94747879bed317e8f1eda/users/aafd5a0da0c4423eabb33fcca0bbef7a/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:23:58.347254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dec611-ee7c-443d-865c-fb9c4d1d5ddc None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:58.347798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dec611-ee7c-443d-865c-fb9c4d1d5ddc None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:58.347798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dec611-ee7c-443d-865c-fb9c4d1d5ddc None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:58.349141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 596/1193] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:23:58.354783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadd01e-fd6f-4c3e-acb7-c76394897e75 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:58.355265 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadd01e-fd6f-4c3e-acb7-c76394897e75 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:58.355365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadd01e-fd6f-4c3e-acb7-c76394897e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:58.375137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-cfadd01e-fd6f-4c3e-acb7-c76394897e75 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:23:58.376362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-cfadd01e-fd6f-4c3e-acb7-c76394897e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:23:58.447854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:58.453141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 598/1194] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:23:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:23:58.476901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dec611-ee7c-443d-865c-fb9c4d1d5ddc None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:58.585539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dec611-ee7c-443d-865c-fb9c4d1d5ddc None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:58.626317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf049677-6f1f-47f0-8794-20e7edbad0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:58.626317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf049677-6f1f-47f0-8794-20e7edbad0ca None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:58.626317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf049677-6f1f-47f0-8794-20e7edbad0ca None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:58.626317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf049677-6f1f-47f0-8794-20e7edbad0ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:58.640622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf049677-6f1f-47f0-8794-20e7edbad0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-bf049677-6f1f-47f0-8794-20e7edbad0ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:23:58.647552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:23:58.648844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:23:58.649720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 597/1195] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:23:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:23:58.655162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadd01e-fd6f-4c3e-acb7-c76394897e75 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:58.763943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadd01e-fd6f-4c3e-acb7-c76394897e75 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:58.801414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5cc5a-783c-48a5-8faa-0a6f0d535929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:58.802063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5cc5a-783c-48a5-8faa-0a6f0d535929 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:58.802350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5cc5a-783c-48a5-8faa-0a6f0d535929 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:58.802623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5cc5a-783c-48a5-8faa-0a6f0d535929 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:58.803337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5cc5a-783c-48a5-8faa-0a6f0d535929 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:58.803646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5cc5a-783c-48a5-8faa-0a6f0d535929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:58.805022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5cc5a-783c-48a5-8faa-0a6f0d535929 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:58.813511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 599/1196] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:23:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:23:58.819799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf049677-6f1f-47f0-8794-20e7edbad0ca None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:58.829345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:58.922971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf049677-6f1f-47f0-8794-20e7edbad0ca None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:58.957650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65754684-1e9b-421a-a763-c3b941629442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:58.958315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754684-1e9b-421a-a763-c3b941629442 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:58.958594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754684-1e9b-421a-a763-c3b941629442 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:58.958866 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754684-1e9b-421a-a763-c3b941629442 None admin] PATH_INFO: `/v3/users/aafd5a0da0c4423eabb33fcca0bbef7a` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:58.960119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65754684-1e9b-421a-a763-c3b941629442 None admin] RBAC: Authorizing `identity:get_user(user_id=aafd5a0da0c4423eabb33fcca0bbef7a)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:58.961542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65754684-1e9b-421a-a763-c3b941629442 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:58.963525 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 598/1197] 10.222.0.73 () {56 vars in 1177 bytes} [Thu Aug 4 21:23:58 2022] GET /identity/v3/users/aafd5a0da0c4423eabb33fcca0bbef7a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:58.970227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5cc5a-783c-48a5-8faa-0a6f0d535929 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:59.032635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:59.073586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5cc5a-783c-48a5-8faa-0a6f0d535929 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:59.107383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c08aec-a08e-4e4b-b1b4-859d703676e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:59.107997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c08aec-a08e-4e4b-b1b4-859d703676e4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:59.109136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c08aec-a08e-4e4b-b1b4-859d703676e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:59.109417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c08aec-a08e-4e4b-b1b4-859d703676e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:59.179658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c08aec-a08e-4e4b-b1b4-859d703676e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:59.181010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c08aec-a08e-4e4b-b1b4-859d703676e4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:59.242469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:59.283537 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 600/1198] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:23:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:23:59.289763 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65754684-1e9b-421a-a763-c3b941629442 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:59.377366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:23:59.403464 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65754684-1e9b-421a-a763-c3b941629442 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:59.439821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbce077-3de5-4877-9c08-6e8f398d7b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:59.441231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbce077-3de5-4877-9c08-6e8f398d7b66 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:59.441231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbce077-3de5-4877-9c08-6e8f398d7b66 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:59.441510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbce077-3de5-4877-9c08-6e8f398d7b66 None admin] PATH_INFO: `/v3/projects/efbde536d0a94747879bed317e8f1eda` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:59.442996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbce077-3de5-4877-9c08-6e8f398d7b66 None admin] RBAC: Authorizing `identity:get_project(project_id=efbde536d0a94747879bed317e8f1eda)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:59.444435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbce077-3de5-4877-9c08-6e8f398d7b66 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:59.446136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 599/1199] 10.222.0.73 () {56 vars in 1186 bytes} [Thu Aug 4 21:23:59 2022] GET /identity/v3/projects/efbde536d0a94747879bed317e8f1eda => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:23:59.452477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c08aec-a08e-4e4b-b1b4-859d703676e4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:23:59.565350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c08aec-a08e-4e4b-b1b4-859d703676e4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:23:59.603675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808da845-dced-4be7-9505-4824041d7893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:23:59.604283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da845-dced-4be7-9505-4824041d7893 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:23:59.604560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da845-dced-4be7-9505-4824041d7893 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:23:59.604829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da845-dced-4be7-9505-4824041d7893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:23:59.605605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808da845-dced-4be7-9505-4824041d7893 None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=efbde536d0a94747879bed317e8f1eda, user.id=aafd5a0da0c4423eabb33fcca0bbef7a) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:23:59.605912 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808da845-dced-4be7-9505-4824041d7893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:23:59.607359 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808da845-dced-4be7-9505-4824041d7893 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:23:59.614379 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 601/1200] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:23:59 2022] GET /identity/v3/role_assignments?user.id=aafd5a0da0c4423eabb33fcca0bbef7a&scope.project.id=efbde536d0a94747879bed317e8f1eda&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:24:09.434791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=29 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:24:09.611266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=87 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:16.371476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3703-cfe6-4ccc-bab1-69dfedab1fb0 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:16.372117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3703-cfe6-4ccc-bab1-69dfedab1fb0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:16.372117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3703-cfe6-4ccc-bab1-69dfedab1fb0 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:16.373361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 600/1201] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:16.378874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea0f19-b9ca-41d7-93d4-6189ce238ce6 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:16.379373 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea0f19-b9ca-41d7-93d4-6189ce238ce6 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:16.379481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea0f19-b9ca-41d7-93d4-6189ce238ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:16.398570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-49ea0f19-b9ca-41d7-93d4-6189ce238ce6 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:16.399781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-49ea0f19-b9ca-41d7-93d4-6189ce238ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:16.478110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 602/1202] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:25:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:25:16.503831 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3703-cfe6-4ccc-bab1-69dfedab1fb0 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:16.621593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3703-cfe6-4ccc-bab1-69dfedab1fb0 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:16.656834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497b97a-aa7c-438d-8b3a-71040d52e762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:16.657476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497b97a-aa7c-438d-8b3a-71040d52e762 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:16.657759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497b97a-aa7c-438d-8b3a-71040d52e762 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:16.658066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497b97a-aa7c-438d-8b3a-71040d52e762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:16.668964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497b97a-aa7c-438d-8b3a-71040d52e762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:16.673054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497b97a-aa7c-438d-8b3a-71040d52e762 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-0497b97a-aa7c-438d-8b3a-71040d52e762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:16.686326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:16.687601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:16.688603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:16.688603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:16.688603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:16.688603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:16.688603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 601/1203] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:16.694258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea0f19-b9ca-41d7-93d4-6189ce238ce6 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:16.797306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea0f19-b9ca-41d7-93d4-6189ce238ce6 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:16.827768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:16.833822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310e4ed-467a-4546-bb89-2def639fca0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:16.834479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310e4ed-467a-4546-bb89-2def639fca0f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:16.834762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310e4ed-467a-4546-bb89-2def639fca0f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:16.835037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310e4ed-467a-4546-bb89-2def639fca0f None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:16.835728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310e4ed-467a-4546-bb89-2def639fca0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:16.836032 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310e4ed-467a-4546-bb89-2def639fca0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:16.837507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310e4ed-467a-4546-bb89-2def639fca0f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:16.852778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 603/1204] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:16.858800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497b97a-aa7c-438d-8b3a-71040d52e762 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:16.913260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:16.960356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497b97a-aa7c-438d-8b3a-71040d52e762 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:16.996556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb11761-cb03-4a8f-8404-6f7897809c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:16.997186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb11761-cb03-4a8f-8404-6f7897809c3e None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:16.997466 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb11761-cb03-4a8f-8404-6f7897809c3e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:16.997740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb11761-cb03-4a8f-8404-6f7897809c3e None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:16.998491 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb11761-cb03-4a8f-8404-6f7897809c3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:16.999911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb11761-cb03-4a8f-8404-6f7897809c3e None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:17.087004 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 602/1205] 10.222.0.73 () {60 vars in 1132 bytes} [Thu Aug 4 21:25:16 2022] POST /identity/v3/users => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:19.203237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419bf52-4149-4ebe-9e6d-9d589a911625 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:19.203774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419bf52-4149-4ebe-9e6d-9d589a911625 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:19.203893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419bf52-4149-4ebe-9e6d-9d589a911625 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.205136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 604/1206] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:19.210881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9006-eff3-4191-a804-ad480ffbba16 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:19.227874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9006-eff3-4191-a804-ad480ffbba16 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:19.227874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9006-eff3-4191-a804-ad480ffbba16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.230307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ce7f9006-eff3-4191-a804-ad480ffbba16 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:19.231523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ce7f9006-eff3-4191-a804-ad480ffbba16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:19.300099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 603/1207] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:25:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:25:19.324439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419bf52-4149-4ebe-9e6d-9d589a911625 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:19.436318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419bf52-4149-4ebe-9e6d-9d589a911625 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:19.470939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d76a7d-7e00-454e-97c1-f9fbe57588bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:19.471495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d76a7d-7e00-454e-97c1-f9fbe57588bf None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:19.471776 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d76a7d-7e00-454e-97c1-f9fbe57588bf None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:19.472049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d76a7d-7e00-454e-97c1-f9fbe57588bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.483445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d76a7d-7e00-454e-97c1-f9fbe57588bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-95d76a7d-7e00-454e-97c1-f9fbe57588bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:25:19.488976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:19.490241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:19.491372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 605/1208] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:25:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:19.496513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f9006-eff3-4191-a804-ad480ffbba16 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:19.539552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:19.598805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f9006-eff3-4191-a804-ad480ffbba16 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:19.634172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcd7cd-b3cf-4447-9cef-e0749f41a500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:19.634797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcd7cd-b3cf-4447-9cef-e0749f41a500 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:19.635075 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcd7cd-b3cf-4447-9cef-e0749f41a500 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:19.635350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcd7cd-b3cf-4447-9cef-e0749f41a500 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.636060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcd7cd-b3cf-4447-9cef-e0749f41a500 None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:19.636362 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcd7cd-b3cf-4447-9cef-e0749f41a500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:19.637720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcd7cd-b3cf-4447-9cef-e0749f41a500 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:19.646457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 604/1209] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:25:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:19.652446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d76a7d-7e00-454e-97c1-f9fbe57588bf None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:19.704219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:19.752683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d76a7d-7e00-454e-97c1-f9fbe57588bf None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:19.788270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a538c-f0d3-4f65-89ea-57af62a5ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:19.788910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a538c-f0d3-4f65-89ea-57af62a5ac48 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:19.789230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a538c-f0d3-4f65-89ea-57af62a5ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:19.789506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a538c-f0d3-4f65-89ea-57af62a5ac48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.799952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a538c-f0d3-4f65-89ea-57af62a5ac48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:19.802056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a538c-f0d3-4f65-89ea-57af62a5ac48 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-a80a538c-f0d3-4f65-89ea-57af62a5ac48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:19.817744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:19.818941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:19.820208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:19.820208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:19.820208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:19.820208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:19.820208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 606/1210] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:19.824928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcd7cd-b3cf-4447-9cef-e0749f41a500 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:19.925377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:19.929266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcd7cd-b3cf-4447-9cef-e0749f41a500 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:19.964621 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a175a-304c-455f-8196-9593d57ecdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:19.965380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a175a-304c-455f-8196-9593d57ecdf6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:19.965847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a175a-304c-455f-8196-9593d57ecdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:19.966160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a175a-304c-455f-8196-9593d57ecdf6 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:19.966930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a175a-304c-455f-8196-9593d57ecdf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:19.967285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a175a-304c-455f-8196-9593d57ecdf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:19.968757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a175a-304c-455f-8196-9593d57ecdf6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:19.984433 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 605/1211] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:19.990408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a538c-f0d3-4f65-89ea-57af62a5ac48 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:20.095847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a538c-f0d3-4f65-89ea-57af62a5ac48 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:20.097316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:20.133462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab738dc-b717-408a-94d5-3a91e2fac8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:20.134146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab738dc-b717-408a-94d5-3a91e2fac8a4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.134430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab738dc-b717-408a-94d5-3a91e2fac8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.134701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab738dc-b717-408a-94d5-3a91e2fac8a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.142462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab738dc-b717-408a-94d5-3a91e2fac8a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:20.143932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab738dc-b717-408a-94d5-3a91e2fac8a4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-6ab738dc-b717-408a-94d5-3a91e2fac8a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:25:20.156568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:20.157834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:20.159125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:20.159125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:20.159125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:25:20.159125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:25:20.159125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:25:20.159125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:25:20.159125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:25:20.159125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:25:20.159125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:20.159125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:20.159125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 607/1212] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:20.164535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a175a-304c-455f-8196-9593d57ecdf6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:20.262347 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a175a-304c-455f-8196-9593d57ecdf6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:20.297836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8efe8e-5eeb-4db0-86d9-0ff13f3e2cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:20.298486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8efe8e-5eeb-4db0-86d9-0ff13f3e2cf4 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.298769 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8efe8e-5eeb-4db0-86d9-0ff13f3e2cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.299044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8efe8e-5eeb-4db0-86d9-0ff13f3e2cf4 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.299833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8efe8e-5eeb-4db0-86d9-0ff13f3e2cf4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:20.300139 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8efe8e-5eeb-4db0-86d9-0ff13f3e2cf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:20.301743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8efe8e-5eeb-4db0-86d9-0ff13f3e2cf4 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:20.318884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 606/1213] 10.222.0.73 () {56 vars in 1143 bytes} [Thu Aug 4 21:25:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:20.324833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab738dc-b717-408a-94d5-3a91e2fac8a4 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:20.430832 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab738dc-b717-408a-94d5-3a91e2fac8a4 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:20.466475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f3948-2004-40bf-8dc2-73d4af78642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:20.467094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f3948-2004-40bf-8dc2-73d4af78642c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.467381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f3948-2004-40bf-8dc2-73d4af78642c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.467649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f3948-2004-40bf-8dc2-73d4af78642c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.500934 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:20.543108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f3948-2004-40bf-8dc2-73d4af78642c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:20.544619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f3948-2004-40bf-8dc2-73d4af78642c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:20.660878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 608/1214] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:25:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:25:20.666878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8efe8e-5eeb-4db0-86d9-0ff13f3e2cf4 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:20.695519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:20.768891 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8efe8e-5eeb-4db0-86d9-0ff13f3e2cf4 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:20.803544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc3aac-0f4a-4619-918a-1931a579aa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:20.804182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc3aac-0f4a-4619-918a-1931a579aa68 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.804461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc3aac-0f4a-4619-918a-1931a579aa68 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.804732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc3aac-0f4a-4619-918a-1931a579aa68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.815089 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc3aac-0f4a-4619-918a-1931a579aa68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:20.817229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc3aac-0f4a-4619-918a-1931a579aa68 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-b1dc3aac-0f4a-4619-918a-1931a579aa68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:20.832783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:20.833852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:20.834805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:20.834805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:20.834805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:20.834805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:20.834805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 607/1215] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:20.839869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f3948-2004-40bf-8dc2-73d4af78642c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:20.906179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:20.945174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f3948-2004-40bf-8dc2-73d4af78642c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:20.978216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedeb26b-c4a1-4865-b1a1-0ac23574faf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:20.978801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedeb26b-c4a1-4865-b1a1-0ac23574faf7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:20.979085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedeb26b-c4a1-4865-b1a1-0ac23574faf7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:20.979357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedeb26b-c4a1-4865-b1a1-0ac23574faf7 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:20.980042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedeb26b-c4a1-4865-b1a1-0ac23574faf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:20.980345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedeb26b-c4a1-4865-b1a1-0ac23574faf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:20.981746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedeb26b-c4a1-4865-b1a1-0ac23574faf7 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:20.995142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 609/1216] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:21.001026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc3aac-0f4a-4619-918a-1931a579aa68 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:21.075794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:21.110448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc3aac-0f4a-4619-918a-1931a579aa68 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:21.148557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c13ba9-9bf3-40d6-82a3-b44c02042a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:21.149242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c13ba9-9bf3-40d6-82a3-b44c02042a58 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:21.149586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c13ba9-9bf3-40d6-82a3-b44c02042a58 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:21.149870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c13ba9-9bf3-40d6-82a3-b44c02042a58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:21.157038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c13ba9-9bf3-40d6-82a3-b44c02042a58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:21.158568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c13ba9-9bf3-40d6-82a3-b44c02042a58 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-d7c13ba9-9bf3-40d6-82a3-b44c02042a58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:25:21.170135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:21.171342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:21.172259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:25:21.172259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:25:21.172259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:25:21.172259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:25:21.172259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:25:21.172259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:25:21.172259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:21.172259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:21.172259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 608/1217] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:25:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:21.178186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedeb26b-c4a1-4865-b1a1-0ac23574faf7 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:21.286510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedeb26b-c4a1-4865-b1a1-0ac23574faf7 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:21.309533 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:21.320816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2406c-cc17-47c0-8cff-f6fa4abcb811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:21.321437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2406c-cc17-47c0-8cff-f6fa4abcb811 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:21.321713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2406c-cc17-47c0-8cff-f6fa4abcb811 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:21.321983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2406c-cc17-47c0-8cff-f6fa4abcb811 None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:21.322763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2406c-cc17-47c0-8cff-f6fa4abcb811 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:21.323065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2406c-cc17-47c0-8cff-f6fa4abcb811 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:21.324497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2406c-cc17-47c0-8cff-f6fa4abcb811 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:21.336305 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 610/1218] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:25:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:21.342458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c13ba9-9bf3-40d6-82a3-b44c02042a58 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:21.448494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c13ba9-9bf3-40d6-82a3-b44c02042a58 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:21.477538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:21.483332 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ab67a-d67c-4ee6-8505-4e3e287fc720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:21.483961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ab67a-d67c-4ee6-8505-4e3e287fc720 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:21.484243 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ab67a-d67c-4ee6-8505-4e3e287fc720 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:21.484558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ab67a-d67c-4ee6-8505-4e3e287fc720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:21.485327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ab67a-d67c-4ee6-8505-4e3e287fc720 None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=fab36b186fa44ee18bf72228fc7ba758) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:21.485639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ab67a-d67c-4ee6-8505-4e3e287fc720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:21.487102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ab67a-d67c-4ee6-8505-4e3e287fc720 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:21.493541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 609/1219] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:25:21 2022] GET /identity/v3/role_assignments?user.id=fab36b186fa44ee18bf72228fc7ba758&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:23.514723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524c82c-9d12-4bee-976a-86b94dc68cde None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:23.515292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524c82c-9d12-4bee-976a-86b94dc68cde None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:23.515292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524c82c-9d12-4bee-976a-86b94dc68cde None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:23.516617 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 611/1220] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:23.522797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e95fe-7aec-47cc-aaaa-3c0f8ff3a56d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:23.523203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e95fe-7aec-47cc-aaaa-3c0f8ff3a56d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:23.523373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e95fe-7aec-47cc-aaaa-3c0f8ff3a56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:23.541529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-8a0e95fe-7aec-47cc-aaaa-3c0f8ff3a56d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:23.542808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8a0e95fe-7aec-47cc-aaaa-3c0f8ff3a56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:23.607651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 610/1221] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:25:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:25:23.630851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524c82c-9d12-4bee-976a-86b94dc68cde None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:23.731312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524c82c-9d12-4bee-976a-86b94dc68cde None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:23.765194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ecc69-04a5-40fe-a30e-0cfba4c6dbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:23.765766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ecc69-04a5-40fe-a30e-0cfba4c6dbec None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:23.766072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ecc69-04a5-40fe-a30e-0cfba4c6dbec None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:23.766348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ecc69-04a5-40fe-a30e-0cfba4c6dbec None admin] PATH_INFO: `/v3/roles/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:23.776701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ecc69-04a5-40fe-a30e-0cfba4c6dbec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-a61ecc69-04a5-40fe-a30e-0cfba4c6dbec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:25:23.781545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:23.782503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:23.783363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 612/1222] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:25:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:23.789029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e95fe-7aec-47cc-aaaa-3c0f8ff3a56d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:23.887945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e95fe-7aec-47cc-aaaa-3c0f8ff3a56d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:23.910256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:23.920644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c623c3a-348d-4f2e-8bca-bd258bd514e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:23.921260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c623c3a-348d-4f2e-8bca-bd258bd514e7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:23.921542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c623c3a-348d-4f2e-8bca-bd258bd514e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:23.921816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c623c3a-348d-4f2e-8bca-bd258bd514e7 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:23.922562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c623c3a-348d-4f2e-8bca-bd258bd514e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:23.922869 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c623c3a-348d-4f2e-8bca-bd258bd514e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:23.924218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c623c3a-348d-4f2e-8bca-bd258bd514e7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:23.933260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 611/1223] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:25:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:23.939394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ecc69-04a5-40fe-a30e-0cfba4c6dbec None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:23.994533 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:24.051632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ecc69-04a5-40fe-a30e-0cfba4c6dbec None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:24.086974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf773d-f088-4a35-9bd4-b10ac48a8f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:24.087620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf773d-f088-4a35-9bd4-b10ac48a8f0c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:24.087901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf773d-f088-4a35-9bd4-b10ac48a8f0c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:24.088171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf773d-f088-4a35-9bd4-b10ac48a8f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:24.099117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf773d-f088-4a35-9bd4-b10ac48a8f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:24.100578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf773d-f088-4a35-9bd4-b10ac48a8f0c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-eecf773d-f088-4a35-9bd4-b10ac48a8f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:24.116546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:24.117826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:24.120451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:24.120451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:24.120451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:24.120451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:24.120451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 613/1224] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:24.125075 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c623c3a-348d-4f2e-8bca-bd258bd514e7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:24.228793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c623c3a-348d-4f2e-8bca-bd258bd514e7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:24.261914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bead4-b6f1-454e-b2e6-2e3c3ea6244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:24.262603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bead4-b6f1-454e-b2e6-2e3c3ea6244f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:24.262884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bead4-b6f1-454e-b2e6-2e3c3ea6244f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:24.263157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bead4-b6f1-454e-b2e6-2e3c3ea6244f None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:24.263845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bead4-b6f1-454e-b2e6-2e3c3ea6244f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:24.264148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bead4-b6f1-454e-b2e6-2e3c3ea6244f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:24.265498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bead4-b6f1-454e-b2e6-2e3c3ea6244f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:24.280928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 612/1225] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:24.296911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf773d-f088-4a35-9bd4-b10ac48a8f0c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:24.389622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:24.396213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf773d-f088-4a35-9bd4-b10ac48a8f0c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:24.429949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13cc7b-4703-4dde-b9b6-3e584134ba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:24.430638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13cc7b-4703-4dde-b9b6-3e584134ba65 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:24.430916 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13cc7b-4703-4dde-b9b6-3e584134ba65 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:24.431188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13cc7b-4703-4dde-b9b6-3e584134ba65 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:24.438340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13cc7b-4703-4dde-b9b6-3e584134ba65 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:24.439816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13cc7b-4703-4dde-b9b6-3e584134ba65 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-aa13cc7b-4703-4dde-b9b6-3e584134ba65 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:25:24.452514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:24.453678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:24.454938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:24.454938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:24.454938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:25:24.454938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:25:24.454938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:25:24.454938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:25:24.454938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:25:24.454938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:25:24.454938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:24.454938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:24.454938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 614/1226] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:24.460325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bead4-b6f1-454e-b2e6-2e3c3ea6244f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:24.507763 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:24.563485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bead4-b6f1-454e-b2e6-2e3c3ea6244f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:24.599113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4aa22-8454-423b-8f62-382bc92960e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:24.599744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4aa22-8454-423b-8f62-382bc92960e7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:24.600027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4aa22-8454-423b-8f62-382bc92960e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:24.600301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4aa22-8454-423b-8f62-382bc92960e7 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:24.601102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4aa22-8454-423b-8f62-382bc92960e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:24.601411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4aa22-8454-423b-8f62-382bc92960e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:24.602873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4aa22-8454-423b-8f62-382bc92960e7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:24.621560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 613/1227] 10.222.0.73 () {56 vars in 1143 bytes} [Thu Aug 4 21:25:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:24.627364 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13cc7b-4703-4dde-b9b6-3e584134ba65 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:24.731854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13cc7b-4703-4dde-b9b6-3e584134ba65 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:24.765728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d987a8-7459-44f8-be62-5f78353cf42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:24.766404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d987a8-7459-44f8-be62-5f78353cf42f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:24.766688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d987a8-7459-44f8-be62-5f78353cf42f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:24.766964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d987a8-7459-44f8-be62-5f78353cf42f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:24.778503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d987a8-7459-44f8-be62-5f78353cf42f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:24.780800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d987a8-7459-44f8-be62-5f78353cf42f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-52d987a8-7459-44f8-be62-5f78353cf42f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:24.794894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:24.795978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:24.796940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:24.796940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:24.796940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:24.796940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:24.796940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 615/1228] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:24.802078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4aa22-8454-423b-8f62-382bc92960e7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:24.903226 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4aa22-8454-423b-8f62-382bc92960e7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:24.905732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:24.938434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ea4a5-9638-42b2-9410-efafa0abd0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:24.939041 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ea4a5-9638-42b2-9410-efafa0abd0c8 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:24.939319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ea4a5-9638-42b2-9410-efafa0abd0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:24.939634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ea4a5-9638-42b2-9410-efafa0abd0c8 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:24.940299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3ea4a5-9638-42b2-9410-efafa0abd0c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:24.940598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3ea4a5-9638-42b2-9410-efafa0abd0c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:24.942545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3ea4a5-9638-42b2-9410-efafa0abd0c8 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:24.954017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 614/1229] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:24.960740 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d987a8-7459-44f8-be62-5f78353cf42f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:24.970803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:25.062744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d987a8-7459-44f8-be62-5f78353cf42f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:25.095539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bd13e-7ead-4af7-b853-f8ed307e93a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:25.096164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bd13e-7ead-4af7-b853-f8ed307e93a7 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:25.096449 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bd13e-7ead-4af7-b853-f8ed307e93a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:25.096758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bd13e-7ead-4af7-b853-f8ed307e93a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:25.103163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58bd13e-7ead-4af7-b853-f8ed307e93a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:25.104585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58bd13e-7ead-4af7-b853-f8ed307e93a7 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-b58bd13e-7ead-4af7-b853-f8ed307e93a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:25:25.116897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:25.117928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:25.124982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:25:25.124982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:25:25.124982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:25:25.124982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:25:25.124982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:25:25.124982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:25:25.124982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:25.124982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:25.124982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 616/1230] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:25:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:25.125479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ea4a5-9638-42b2-9410-efafa0abd0c8 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:25.222316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ea4a5-9638-42b2-9410-efafa0abd0c8 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:25.258539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc62cc-c80d-41b4-82d1-ba560171561f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:25.259164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc62cc-c80d-41b4-82d1-ba560171561f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:25.259448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc62cc-c80d-41b4-82d1-ba560171561f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:25.259723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc62cc-c80d-41b4-82d1-ba560171561f None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:25.260468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc62cc-c80d-41b4-82d1-ba560171561f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:25.260777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc62cc-c80d-41b4-82d1-ba560171561f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:25.262219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc62cc-c80d-41b4-82d1-ba560171561f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:25.275325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 615/1231] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:25:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:25.281454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bd13e-7ead-4af7-b853-f8ed307e93a7 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:25.346424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:25.381825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bd13e-7ead-4af7-b853-f8ed307e93a7 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:25.419979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399fd05-c2d9-4645-941b-4107e96015ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:25.421033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399fd05-c2d9-4645-941b-4107e96015ff None admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:25.421484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399fd05-c2d9-4645-941b-4107e96015ff None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:25.421951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399fd05-c2d9-4645-941b-4107e96015ff None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/fab36b186fa44ee18bf72228fc7ba758/roles/280f24425d0a4b16951d30c20d7290f9` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:25.438353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399fd05-c2d9-4645-941b-4107e96015ff None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=fab36b186fa44ee18bf72228fc7ba758, role_id=280f24425d0a4b16951d30c20d7290f9)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:25.439813 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399fd05-c2d9-4645-941b-4107e96015ff None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:25.492225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 617/1232] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:25:25 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/fab36b186fa44ee18bf72228fc7ba758/roles/280f24425d0a4b16951d30c20d7290f9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:27.530296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f444501-d330-456b-84f7-e53f73d4c9e9 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:27.530850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f444501-d330-456b-84f7-e53f73d4c9e9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:27.530850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f444501-d330-456b-84f7-e53f73d4c9e9 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:27.532184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 616/1233] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:27.538260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d120b-f915-4b3f-80af-17ed1592a453 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:27.538720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d120b-f915-4b3f-80af-17ed1592a453 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:27.538825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d120b-f915-4b3f-80af-17ed1592a453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:27.557474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-529d120b-f915-4b3f-80af-17ed1592a453 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:27.558721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-529d120b-f915-4b3f-80af-17ed1592a453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:27.628023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 618/1234] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:25:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:25:27.651500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f444501-d330-456b-84f7-e53f73d4c9e9 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:27.677309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:27.764760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f444501-d330-456b-84f7-e53f73d4c9e9 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:27.801055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc52a1-1de8-48dd-a2c4-05abe02a951f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:27.801561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc52a1-1de8-48dd-a2c4-05abe02a951f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:27.801842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc52a1-1de8-48dd-a2c4-05abe02a951f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:27.802143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc52a1-1de8-48dd-a2c4-05abe02a951f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:27.816827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc52a1-1de8-48dd-a2c4-05abe02a951f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-47dc52a1-1de8-48dd-a2c4-05abe02a951f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:25:27.821802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:27.823003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:27.824053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 617/1235] 10.222.0.73 () {56 vars in 1102 bytes} [Thu Aug 4 21:25:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:27.829532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d120b-f915-4b3f-80af-17ed1592a453 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:27.877190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:27.929760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d120b-f915-4b3f-80af-17ed1592a453 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:27.963513 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fdb67-080b-4de3-a9f1-b65985514587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:27.964134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fdb67-080b-4de3-a9f1-b65985514587 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:27.964417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fdb67-080b-4de3-a9f1-b65985514587 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:27.964691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fdb67-080b-4de3-a9f1-b65985514587 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:27.965401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fdb67-080b-4de3-a9f1-b65985514587 None admin] RBAC: Adding query filter params (name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:27.965715 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fdb67-080b-4de3-a9f1-b65985514587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:27.967121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fdb67-080b-4de3-a9f1-b65985514587 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:27.975518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 619/1236] 10.222.0.73 () {56 vars in 1103 bytes} [Thu Aug 4 21:25:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:27.981415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc52a1-1de8-48dd-a2c4-05abe02a951f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.065126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:28.081236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc52a1-1de8-48dd-a2c4-05abe02a951f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:28.115014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6643c1c-e52a-4824-8562-80d884fc826a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:28.115643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6643c1c-e52a-4824-8562-80d884fc826a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.115920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6643c1c-e52a-4824-8562-80d884fc826a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.116209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6643c1c-e52a-4824-8562-80d884fc826a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.126094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6643c1c-e52a-4824-8562-80d884fc826a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:28.127528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6643c1c-e52a-4824-8562-80d884fc826a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-c6643c1c-e52a-4824-8562-80d884fc826a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:28.142373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:28.143465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:28.144607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:28.144607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:28.144607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:28.144607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:28.144607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 618/1237] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:28.150267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fdb67-080b-4de3-a9f1-b65985514587 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.252862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fdb67-080b-4de3-a9f1-b65985514587 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:28.262505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:28.287120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36e3ad-65d8-4017-b328-2c58c80f6295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:28.287734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36e3ad-65d8-4017-b328-2c58c80f6295 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.288019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36e3ad-65d8-4017-b328-2c58c80f6295 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.288289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36e3ad-65d8-4017-b328-2c58c80f6295 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.288976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36e3ad-65d8-4017-b328-2c58c80f6295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:28.289281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36e3ad-65d8-4017-b328-2c58c80f6295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:28.290670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36e3ad-65d8-4017-b328-2c58c80f6295 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:28.302730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 620/1238] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:28.309345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6643c1c-e52a-4824-8562-80d884fc826a None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.414482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6643c1c-e52a-4824-8562-80d884fc826a None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:28.448596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a11bb1-35d1-47d7-8518-d3a77ccd2cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:28.449230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a11bb1-35d1-47d7-8518-d3a77ccd2cf9 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.449514 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a11bb1-35d1-47d7-8518-d3a77ccd2cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.449818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a11bb1-35d1-47d7-8518-d3a77ccd2cf9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.452473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:28.457536 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a11bb1-35d1-47d7-8518-d3a77ccd2cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:28.458994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a11bb1-35d1-47d7-8518-d3a77ccd2cf9 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-66a11bb1-35d1-47d7-8518-d3a77ccd2cf9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:25:28.470279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:28.471457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:28.473373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:28.473373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:28.473373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:25:28.473373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:25:28.473373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:25:28.473373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:25:28.473373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:25:28.473373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:25:28.473373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:28.473373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:28.473373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 619/1239] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:28.478629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36e3ad-65d8-4017-b328-2c58c80f6295 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.578809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36e3ad-65d8-4017-b328-2c58c80f6295 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:28.612490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdef7af-699d-44c4-b398-06f377cf069c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:28.613971 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdef7af-699d-44c4-b398-06f377cf069c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.618781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdef7af-699d-44c4-b398-06f377cf069c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.618781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdef7af-699d-44c4-b398-06f377cf069c None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.618781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdef7af-699d-44c4-b398-06f377cf069c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:28.618781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdef7af-699d-44c4-b398-06f377cf069c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:28.618781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdef7af-699d-44c4-b398-06f377cf069c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:28.634943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 621/1240] 10.222.0.73 () {56 vars in 1143 bytes} [Thu Aug 4 21:25:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:28.641559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a11bb1-35d1-47d7-8518-d3a77ccd2cf9 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.745374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a11bb1-35d1-47d7-8518-d3a77ccd2cf9 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:28.782703 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159b80c-7f9a-4736-9fb3-80988577f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:28.783333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159b80c-7f9a-4736-9fb3-80988577f13f None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:28.783616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159b80c-7f9a-4736-9fb3-80988577f13f None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:28.783894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159b80c-7f9a-4736-9fb3-80988577f13f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:28.849680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:28.856378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159b80c-7f9a-4736-9fb3-80988577f13f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:28.857868 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159b80c-7f9a-4736-9fb3-80988577f13f None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:28.957721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 620/1241] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:25:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:25:28.963672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdef7af-699d-44c4-b398-06f377cf069c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:28.993378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:29.070899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdef7af-699d-44c4-b398-06f377cf069c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:29.110999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a7052-4747-40be-b874-f8d2d8abdb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:29.111659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a7052-4747-40be-b874-f8d2d8abdb82 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:29.111982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a7052-4747-40be-b874-f8d2d8abdb82 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:29.112296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a7052-4747-40be-b874-f8d2d8abdb82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:29.123085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a7052-4747-40be-b874-f8d2d8abdb82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:29.124624 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a7052-4747-40be-b874-f8d2d8abdb82 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-015a7052-4747-40be-b874-f8d2d8abdb82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:29.139734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:29.140819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:29.141919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:29.141919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:29.141919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:29.141919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:29.141919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 622/1242] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:25:29.147436 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159b80c-7f9a-4736-9fb3-80988577f13f None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:29.246182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:29.251949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159b80c-7f9a-4736-9fb3-80988577f13f None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:29.287860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b95f3-1cab-465c-a889-e8fb7faa246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:29.288476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b95f3-1cab-465c-a889-e8fb7faa246b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:29.288757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b95f3-1cab-465c-a889-e8fb7faa246b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:29.289030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b95f3-1cab-465c-a889-e8fb7faa246b None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:29.289731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b95f3-1cab-465c-a889-e8fb7faa246b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:29.290060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b95f3-1cab-465c-a889-e8fb7faa246b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:29.291421 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b95f3-1cab-465c-a889-e8fb7faa246b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:29.302860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 621/1243] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:29.308724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a7052-4747-40be-b874-f8d2d8abdb82 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:29.386785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:29.417312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a7052-4747-40be-b874-f8d2d8abdb82 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:29.450529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4e3be-1148-42f6-939f-dac34b716eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:29.451177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4e3be-1148-42f6-939f-dac34b716eca None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:29.451462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4e3be-1148-42f6-939f-dac34b716eca None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:29.451767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4e3be-1148-42f6-939f-dac34b716eca None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:29.459393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4e3be-1148-42f6-939f-dac34b716eca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:29.460975 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4e3be-1148-42f6-939f-dac34b716eca None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-d7f4e3be-1148-42f6-939f-dac34b716eca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:25:29.473161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:29.474809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:29.476420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:25:29.476420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:25:29.476420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:25:29.476420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:25:29.476420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:25:29.476420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:25:29.476420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:29.476420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:29.476420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 623/1244] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:25:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:29.480592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b95f3-1cab-465c-a889-e8fb7faa246b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:29.585955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b95f3-1cab-465c-a889-e8fb7faa246b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:29.624086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0b481-ee3c-4a7d-a3b6-2314a07a230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:29.624700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0b481-ee3c-4a7d-a3b6-2314a07a230d None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:29.624980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0b481-ee3c-4a7d-a3b6-2314a07a230d None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:29.625256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0b481-ee3c-4a7d-a3b6-2314a07a230d None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:29.625998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0b481-ee3c-4a7d-a3b6-2314a07a230d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:29.626332 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0b481-ee3c-4a7d-a3b6-2314a07a230d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:29.627742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0b481-ee3c-4a7d-a3b6-2314a07a230d None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:29.643716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 622/1245] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:25:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:29.646547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4e3be-1148-42f6-939f-dac34b716eca None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:29.751229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4e3be-1148-42f6-939f-dac34b716eca None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:29.785883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ae1d7-8669-4fa5-a01a-27f6c0c8f058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:29.786572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ae1d7-8669-4fa5-a01a-27f6c0c8f058 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:29.786857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ae1d7-8669-4fa5-a01a-27f6c0c8f058 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:29.787151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ae1d7-8669-4fa5-a01a-27f6c0c8f058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:29.788069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ae1d7-8669-4fa5-a01a-27f6c0c8f058 None admin] RBAC: Adding query filter params (role.id=280f24425d0a4b16951d30c20d7290f9, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=fab36b186fa44ee18bf72228fc7ba758) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:29.788383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ae1d7-8669-4fa5-a01a-27f6c0c8f058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:29.789834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ae1d7-8669-4fa5-a01a-27f6c0c8f058 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:29.796296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 624/1246] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:25:29 2022] GET /identity/v3/role_assignments?user.id=fab36b186fa44ee18bf72228fc7ba758&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=280f24425d0a4b16951d30c20d7290f9 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:31.928025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba69649-938b-4c5e-8ff1-4a72071144a4 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:31.928534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba69649-938b-4c5e-8ff1-4a72071144a4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:31.928534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba69649-938b-4c5e-8ff1-4a72071144a4 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:31.929981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 623/1247] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:31.935343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c296f-8010-45fc-9a6e-f9b347ed8e50 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:31.935800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c296f-8010-45fc-9a6e-f9b347ed8e50 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:31.935924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c296f-8010-45fc-9a6e-f9b347ed8e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:31.943027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:31.955530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-fb5c296f-8010-45fc-9a6e-f9b347ed8e50 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:31.957446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-fb5c296f-8010-45fc-9a6e-f9b347ed8e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:32.029937 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 625/1248] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:25:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:25:32.052753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba69649-938b-4c5e-8ff1-4a72071144a4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:32.071974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:32.175221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba69649-938b-4c5e-8ff1-4a72071144a4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:32.212005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef13ce-0b57-4cf5-b1eb-c3cee478c36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:32.213105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef13ce-0b57-4cf5-b1eb-c3cee478c36e None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:32.213513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef13ce-0b57-4cf5-b1eb-c3cee478c36e None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:32.213788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef13ce-0b57-4cf5-b1eb-c3cee478c36e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:32.228564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef13ce-0b57-4cf5-b1eb-c3cee478c36e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-b5ef13ce-0b57-4cf5-b1eb-c3cee478c36e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:25:32.233415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:32.235096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:32.236729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 624/1249] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:25:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:32.243404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c296f-8010-45fc-9a6e-f9b347ed8e50 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:32.361924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c296f-8010-45fc-9a6e-f9b347ed8e50 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:32.400847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:32.407122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd94d0-475e-4397-9a88-d79efb4d1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:32.407979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd94d0-475e-4397-9a88-d79efb4d1d2c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:32.408263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd94d0-475e-4397-9a88-d79efb4d1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:32.408535 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd94d0-475e-4397-9a88-d79efb4d1d2c None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:32.409431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd94d0-475e-4397-9a88-d79efb4d1d2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:32.410035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd94d0-475e-4397-9a88-d79efb4d1d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:32.411859 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd94d0-475e-4397-9a88-d79efb4d1d2c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:32.421924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 626/1250] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:25:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:32.428594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef13ce-0b57-4cf5-b1eb-c3cee478c36e None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:32.508416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:32.531386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef13ce-0b57-4cf5-b1eb-c3cee478c36e None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:32.568275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e65365-8ac7-4dd2-8afe-6dd23240e9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:32.568916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e65365-8ac7-4dd2-8afe-6dd23240e9be None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:32.569199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e65365-8ac7-4dd2-8afe-6dd23240e9be None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:32.569474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e65365-8ac7-4dd2-8afe-6dd23240e9be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:32.580292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e65365-8ac7-4dd2-8afe-6dd23240e9be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:32.581805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e65365-8ac7-4dd2-8afe-6dd23240e9be None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-30e65365-8ac7-4dd2-8afe-6dd23240e9be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:32.596335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:32.598770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:32.600094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:32.600094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:32.600094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:32.600094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:32.600094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 625/1251] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:32.603922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd94d0-475e-4397-9a88-d79efb4d1d2c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:32.706230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd94d0-475e-4397-9a88-d79efb4d1d2c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:32.741377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312c882-7c9a-425f-a811-b7baf28f1c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:32.741993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312c882-7c9a-425f-a811-b7baf28f1c3b None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:32.742301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312c882-7c9a-425f-a811-b7baf28f1c3b None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:32.742573 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312c882-7c9a-425f-a811-b7baf28f1c3b None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:32.743269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312c882-7c9a-425f-a811-b7baf28f1c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:32.743574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312c882-7c9a-425f-a811-b7baf28f1c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:32.744945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312c882-7c9a-425f-a811-b7baf28f1c3b None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:32.756904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 627/1252] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:32.762810 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e65365-8ac7-4dd2-8afe-6dd23240e9be None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:32.868888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e65365-8ac7-4dd2-8afe-6dd23240e9be None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:32.895272 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:32.902881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6c8a3-0752-4bf4-8269-dc3fd64f795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:32.903523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6c8a3-0752-4bf4-8269-dc3fd64f795c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:32.903811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6c8a3-0752-4bf4-8269-dc3fd64f795c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:32.904086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6c8a3-0752-4bf4-8269-dc3fd64f795c None admin] PATH_INFO: `/v3/users/placement` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:32.912332 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6c8a3-0752-4bf4-8269-dc3fd64f795c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:32.917869 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6c8a3-0752-4bf4-8269-dc3fd64f795c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-d8b6c8a3-0752-4bf4-8269-dc3fd64f795c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:25:32.925613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:32.930925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:32.932598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:32.932598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:32.932598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:25:32.932598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:25:32.932598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:25:32.932598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:25:32.932598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:25:32.932598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:25:32.932598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:32.932598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:32.932598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 626/1253] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:32.933455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312c882-7c9a-425f-a811-b7baf28f1c3b None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:32.991219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:33.038288 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312c882-7c9a-425f-a811-b7baf28f1c3b None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:33.072764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe8ad8-fa73-451f-bc26-2f70018a3b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:33.073401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe8ad8-fa73-451f-bc26-2f70018a3b4f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:33.073685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe8ad8-fa73-451f-bc26-2f70018a3b4f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:33.073960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe8ad8-fa73-451f-bc26-2f70018a3b4f None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:33.074795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe8ad8-fa73-451f-bc26-2f70018a3b4f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:33.075106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe8ad8-fa73-451f-bc26-2f70018a3b4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:33.076536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe8ad8-fa73-451f-bc26-2f70018a3b4f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:33.094485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 628/1254] 10.222.0.73 () {56 vars in 1143 bytes} [Thu Aug 4 21:25:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:25:33.100965 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6c8a3-0752-4bf4-8269-dc3fd64f795c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:33.205048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6c8a3-0752-4bf4-8269-dc3fd64f795c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:33.240687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503a92a-4f4e-440a-ab51-0091d85cf6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:33.241332 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503a92a-4f4e-440a-ab51-0091d85cf6de None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:33.241615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503a92a-4f4e-440a-ab51-0091d85cf6de None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:33.241960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503a92a-4f4e-440a-ab51-0091d85cf6de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:33.302995 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:33.314737 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503a92a-4f4e-440a-ab51-0091d85cf6de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:33.316729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503a92a-4f4e-440a-ab51-0091d85cf6de None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:33.413291 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 627/1255] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:25:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:25:33.419293 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe8ad8-fa73-451f-bc26-2f70018a3b4f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:33.522169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe8ad8-fa73-451f-bc26-2f70018a3b4f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:33.536247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:33.556978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ab3c-218d-4196-aea1-e1b566d90e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:33.557536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ab3c-218d-4196-aea1-e1b566d90e48 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:33.557812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ab3c-218d-4196-aea1-e1b566d90e48 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:33.558115 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ab3c-218d-4196-aea1-e1b566d90e48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:33.569338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323ab3c-218d-4196-aea1-e1b566d90e48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:33.570828 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323ab3c-218d-4196-aea1-e1b566d90e48 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-f323ab3c-218d-4196-aea1-e1b566d90e48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:33.586452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:33.587701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:33.588807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:33.588807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:33.588807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:33.588807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:33.588807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 629/1256] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:33.593479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503a92a-4f4e-440a-ab51-0091d85cf6de None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:33.683235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:33.695295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503a92a-4f4e-440a-ab51-0091d85cf6de None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:33.728990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284ba47-fd1b-4979-889b-f45a21bc94c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:33.729654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284ba47-fd1b-4979-889b-f45a21bc94c2 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:33.729934 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284ba47-fd1b-4979-889b-f45a21bc94c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:33.730239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284ba47-fd1b-4979-889b-f45a21bc94c2 None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:33.730950 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284ba47-fd1b-4979-889b-f45a21bc94c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:33.731282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284ba47-fd1b-4979-889b-f45a21bc94c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:33.732724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284ba47-fd1b-4979-889b-f45a21bc94c2 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:33.745037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 628/1257] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:33.750929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ab3c-218d-4196-aea1-e1b566d90e48 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:33.852111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ab3c-218d-4196-aea1-e1b566d90e48 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:33.886316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e84063-bb62-4c89-ba53-ab331bc5d36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:33.886956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e84063-bb62-4c89-ba53-ab331bc5d36f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:33.887236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e84063-bb62-4c89-ba53-ab331bc5d36f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:33.887509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e84063-bb62-4c89-ba53-ab331bc5d36f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:33.894792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e84063-bb62-4c89-ba53-ab331bc5d36f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:33.896233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e84063-bb62-4c89-ba53-ab331bc5d36f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-a5e84063-bb62-4c89-ba53-ab331bc5d36f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:25:33.907673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:33.908714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:33.910833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:25:33.910833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:25:33.910833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:25:33.910833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:25:33.910833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:25:33.910833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:25:33.910833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:33.910833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:33.910833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 630/1258] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:25:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:33.915632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284ba47-fd1b-4979-889b-f45a21bc94c2 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:34.021263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284ba47-fd1b-4979-889b-f45a21bc94c2 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:34.055176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c8cec-8fa0-4583-9f4f-0134f87ba10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:34.055802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c8cec-8fa0-4583-9f4f-0134f87ba10a None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:34.056086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c8cec-8fa0-4583-9f4f-0134f87ba10a None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:34.056434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c8cec-8fa0-4583-9f4f-0134f87ba10a None admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:34.057182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c8cec-8fa0-4583-9f4f-0134f87ba10a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:34.057492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c8cec-8fa0-4583-9f4f-0134f87ba10a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:34.058956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c8cec-8fa0-4583-9f4f-0134f87ba10a None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:34.072588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 629/1259] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:25:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:34.079219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e84063-bb62-4c89-ba53-ab331bc5d36f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:34.108295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:34.187493 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e84063-bb62-4c89-ba53-ab331bc5d36f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:34.223242 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f16d3-5fb7-483f-8f78-c4422b244be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:34.223853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f16d3-5fb7-483f-8f78-c4422b244be4 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:34.224137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f16d3-5fb7-483f-8f78-c4422b244be4 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:34.224413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f16d3-5fb7-483f-8f78-c4422b244be4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:34.225281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f16d3-5fb7-483f-8f78-c4422b244be4 None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=fab36b186fa44ee18bf72228fc7ba758) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:34.225587 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f16d3-5fb7-483f-8f78-c4422b244be4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:34.227112 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f16d3-5fb7-483f-8f78-c4422b244be4 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:34.232800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 631/1260] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:25:34 2022] GET /identity/v3/role_assignments?user.id=fab36b186fa44ee18bf72228fc7ba758&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:36.324175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e785421-1dbf-4186-bf44-876c13b2f9a6 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:36.324740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e785421-1dbf-4186-bf44-876c13b2f9a6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:36.324740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e785421-1dbf-4186-bf44-876c13b2f9a6 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:36.326080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 630/1261] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:36.331638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baee7a-2158-4aac-a758-4ba6a7720c5d None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:36.332142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baee7a-2158-4aac-a758-4ba6a7720c5d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:36.332695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baee7a-2158-4aac-a758-4ba6a7720c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:36.351719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-25baee7a-2158-4aac-a758-4ba6a7720c5d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:36.352983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-25baee7a-2158-4aac-a758-4ba6a7720c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:36.425081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 632/1262] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:25:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:25:36.448789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e785421-1dbf-4186-bf44-876c13b2f9a6 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:36.460378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:36.556694 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e785421-1dbf-4186-bf44-876c13b2f9a6 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:36.592105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb223ee-5981-40a4-bb8c-c15e9b22eb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:36.592652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb223ee-5981-40a4-bb8c-c15e9b22eb8b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:36.592931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb223ee-5981-40a4-bb8c-c15e9b22eb8b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:36.593212 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb223ee-5981-40a4-bb8c-c15e9b22eb8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:36.613990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb223ee-5981-40a4-bb8c-c15e9b22eb8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-5cb223ee-5981-40a4-bb8c-c15e9b22eb8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:25:36.623438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:36.624767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:36.625826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 631/1263] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:25:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:36.632620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25baee7a-2158-4aac-a758-4ba6a7720c5d None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:36.654068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:36.747735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25baee7a-2158-4aac-a758-4ba6a7720c5d None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:36.786586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad44561-7baf-48aa-8178-bd3c9554dab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:36.787213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad44561-7baf-48aa-8178-bd3c9554dab6 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:36.787492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad44561-7baf-48aa-8178-bd3c9554dab6 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:36.787765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad44561-7baf-48aa-8178-bd3c9554dab6 None admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:36.788481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad44561-7baf-48aa-8178-bd3c9554dab6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:36.788787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad44561-7baf-48aa-8178-bd3c9554dab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:36.790187 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad44561-7baf-48aa-8178-bd3c9554dab6 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:36.798366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 633/1264] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:25:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:25:36.804593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb223ee-5981-40a4-bb8c-c15e9b22eb8b None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:36.875626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:36.910579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb223ee-5981-40a4-bb8c-c15e9b22eb8b None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:36.947070 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb9542-387d-4275-99e2-68f78355a09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:36.947778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9542-387d-4275-99e2-68f78355a09c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:36.947992 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9542-387d-4275-99e2-68f78355a09c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:36.948267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9542-387d-4275-99e2-68f78355a09c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:36.960642 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb9542-387d-4275-99e2-68f78355a09c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:36.962107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb9542-387d-4275-99e2-68f78355a09c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-40eb9542-387d-4275-99e2-68f78355a09c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:36.976063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:36.977218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:36.978530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:36.978530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:36.978530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:36.978530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:36.978530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 632/1265] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:36.983225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad44561-7baf-48aa-8178-bd3c9554dab6 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:37.066911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:37.087426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad44561-7baf-48aa-8178-bd3c9554dab6 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:37.126219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1acd4c-134e-43ef-8965-8b2389f4dc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:37.126808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1acd4c-134e-43ef-8965-8b2389f4dc95 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:37.127117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1acd4c-134e-43ef-8965-8b2389f4dc95 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:37.127399 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1acd4c-134e-43ef-8965-8b2389f4dc95 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:37.128093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1acd4c-134e-43ef-8965-8b2389f4dc95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:37.128605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1acd4c-134e-43ef-8965-8b2389f4dc95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:37.130123 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1acd4c-134e-43ef-8965-8b2389f4dc95 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:37.141505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 634/1266] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:25:37.147848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb9542-387d-4275-99e2-68f78355a09c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:37.267241 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb9542-387d-4275-99e2-68f78355a09c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:37.288828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:37.303059 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735c9d2-ecac-478f-b510-3a0bad86da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:37.303765 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735c9d2-ecac-478f-b510-3a0bad86da79 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:37.304051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735c9d2-ecac-478f-b510-3a0bad86da79 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:37.304324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735c9d2-ecac-478f-b510-3a0bad86da79 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:37.312091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735c9d2-ecac-478f-b510-3a0bad86da79 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:37.313545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735c9d2-ecac-478f-b510-3a0bad86da79 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-e735c9d2-ecac-478f-b510-3a0bad86da79 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:25:37.324693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:37.326300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:37.327406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:37.327406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:37.327406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:25:37.327406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:25:37.327406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:25:37.327406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:25:37.327406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:25:37.327406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:25:37.327406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:37.327406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:37.327406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 633/1267] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:37.332667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1acd4c-134e-43ef-8965-8b2389f4dc95 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:37.444924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1acd4c-134e-43ef-8965-8b2389f4dc95 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:37.484179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924f360-7373-4953-82cf-ca70ef735e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:37.484798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924f360-7373-4953-82cf-ca70ef735e0f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:37.485080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924f360-7373-4953-82cf-ca70ef735e0f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:37.485357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924f360-7373-4953-82cf-ca70ef735e0f None admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:37.486291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924f360-7373-4953-82cf-ca70ef735e0f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:37.486491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924f360-7373-4953-82cf-ca70ef735e0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:37.488318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924f360-7373-4953-82cf-ca70ef735e0f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:37.497939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:37.508085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 635/1268] 10.222.0.73 () {56 vars in 1143 bytes} [Thu Aug 4 21:25:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:37.514038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735c9d2-ecac-478f-b510-3a0bad86da79 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:37.618496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735c9d2-ecac-478f-b510-3a0bad86da79 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:37.653011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9755d-7dd6-4c3c-a45d-90b9d30cbdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:37.653647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9755d-7dd6-4c3c-a45d-90b9d30cbdda None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:37.653927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9755d-7dd6-4c3c-a45d-90b9d30cbdda None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:37.654229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9755d-7dd6-4c3c-a45d-90b9d30cbdda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:37.665478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9755d-7dd6-4c3c-a45d-90b9d30cbdda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:37.666964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9755d-7dd6-4c3c-a45d-90b9d30cbdda None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-1aa9755d-7dd6-4c3c-a45d-90b9d30cbdda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:37.681601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:37.683219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:37.684268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:37.684268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:37.684268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:37.684268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:37.684268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 634/1269] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:37.688752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924f360-7373-4953-82cf-ca70ef735e0f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:37.802993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924f360-7373-4953-82cf-ca70ef735e0f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:37.839711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c41a8-53aa-4ebd-904e-18036459426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:37.840369 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c41a8-53aa-4ebd-904e-18036459426c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:37.840647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c41a8-53aa-4ebd-904e-18036459426c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:37.840917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c41a8-53aa-4ebd-904e-18036459426c None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:37.841606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c41a8-53aa-4ebd-904e-18036459426c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:37.842041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c41a8-53aa-4ebd-904e-18036459426c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:37.843419 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c41a8-53aa-4ebd-904e-18036459426c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:37.855879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 636/1270] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:37.861836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9755d-7dd6-4c3c-a45d-90b9d30cbdda None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:37.897577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:37.977825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9755d-7dd6-4c3c-a45d-90b9d30cbdda None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:38.017581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd9b15-74d5-42e0-a42a-af1127115b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:38.018256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd9b15-74d5-42e0-a42a-af1127115b07 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:38.018541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd9b15-74d5-42e0-a42a-af1127115b07 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:38.018890 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd9b15-74d5-42e0-a42a-af1127115b07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:38.026423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd9b15-74d5-42e0-a42a-af1127115b07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:38.027921 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd9b15-74d5-42e0-a42a-af1127115b07 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-badd9b15-74d5-42e0-a42a-af1127115b07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:25:38.039083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:38.040151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:38.041331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:25:38.041331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:25:38.041331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:25:38.041331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:25:38.041331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:25:38.041331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:25:38.041331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:38.041331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:38.041331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 635/1271] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:25:38.046263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c41a8-53aa-4ebd-904e-18036459426c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:38.142459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:38.203841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c41a8-53aa-4ebd-904e-18036459426c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:38.244705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18312f1c-3d54-43fc-b759-2d7e0edf525d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:38.245749 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18312f1c-3d54-43fc-b759-2d7e0edf525d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:38.246253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18312f1c-3d54-43fc-b759-2d7e0edf525d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:38.246702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18312f1c-3d54-43fc-b759-2d7e0edf525d None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:38.247912 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18312f1c-3d54-43fc-b759-2d7e0edf525d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:38.248427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18312f1c-3d54-43fc-b759-2d7e0edf525d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:38.251247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18312f1c-3d54-43fc-b759-2d7e0edf525d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:38.265886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 637/1272] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:25:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:38.272776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd9b15-74d5-42e0-a42a-af1127115b07 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:38.387392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:38.397403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd9b15-74d5-42e0-a42a-af1127115b07 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:38.441161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e236d9c-ab3e-4ad0-9ebf-ec595cfccb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:38.441796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e236d9c-ab3e-4ad0-9ebf-ec595cfccb10 None admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:38.442717 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e236d9c-ab3e-4ad0-9ebf-ec595cfccb10 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:38.443005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e236d9c-ab3e-4ad0-9ebf-ec595cfccb10 None admin] PATH_INFO: `/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/fab36b186fa44ee18bf72228fc7ba758/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:38.444978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e236d9c-ab3e-4ad0-9ebf-ec595cfccb10 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a470d4b3c4a96abc633051a2f74f1, user_id=fab36b186fa44ee18bf72228fc7ba758, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:38.446439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e236d9c-ab3e-4ad0-9ebf-ec595cfccb10 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:38.565982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 636/1273] 10.222.0.73 () {58 vars in 1439 bytes} [Thu Aug 4 21:25:38 2022] PUT /identity/v3/projects/ca8a470d4b3c4a96abc633051a2f74f1/users/fab36b186fa44ee18bf72228fc7ba758/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:25:40.611296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9e02f-c9c8-4596-94b5-311ac938bf18 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:40.611819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9e02f-c9c8-4596-94b5-311ac938bf18 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:40.611819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9e02f-c9c8-4596-94b5-311ac938bf18 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:40.613577 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 638/1274] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:40.619105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd57aee-73f4-4980-8b4d-d2c7d96c7fd5 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:40.619105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd57aee-73f4-4980-8b4d-d2c7d96c7fd5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:40.619614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd57aee-73f4-4980-8b4d-d2c7d96c7fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:40.638561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-8dd57aee-73f4-4980-8b4d-d2c7d96c7fd5 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:40.639767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8dd57aee-73f4-4980-8b4d-d2c7d96c7fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:40.711427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 637/1275] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:25:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:25:40.734220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9e02f-c9c8-4596-94b5-311ac938bf18 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:40.844151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9e02f-c9c8-4596-94b5-311ac938bf18 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:40.863524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:40.882337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5355ad-aef8-41a4-bcbb-eac93423ea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:40.882986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5355ad-aef8-41a4-bcbb-eac93423ea7c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:40.883264 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5355ad-aef8-41a4-bcbb-eac93423ea7c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:40.883536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5355ad-aef8-41a4-bcbb-eac93423ea7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:40.893779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5355ad-aef8-41a4-bcbb-eac93423ea7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-8b5355ad-aef8-41a4-bcbb-eac93423ea7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 21:25:40.898720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:40.906637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:40.907793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 639/1276] 10.222.0.73 () {56 vars in 1096 bytes} [Thu Aug 4 21:25:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:40.909172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd57aee-73f4-4980-8b4d-d2c7d96c7fd5 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:40.998287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:41.017100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd57aee-73f4-4980-8b4d-d2c7d96c7fd5 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:41.054715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f3ec4-2273-45dd-9513-a766d4ad6489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:41.055365 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f3ec4-2273-45dd-9513-a766d4ad6489 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:41.055652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f3ec4-2273-45dd-9513-a766d4ad6489 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:41.055931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f3ec4-2273-45dd-9513-a766d4ad6489 None admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:41.056652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f3ec4-2273-45dd-9513-a766d4ad6489 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:41.057096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f3ec4-2273-45dd-9513-a766d4ad6489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:41.058497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f3ec4-2273-45dd-9513-a766d4ad6489 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:41.067418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 638/1277] 10.222.0.73 () {56 vars in 1099 bytes} [Thu Aug 4 21:25:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:25:41.073289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5355ad-aef8-41a4-bcbb-eac93423ea7c None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:41.183719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5355ad-aef8-41a4-bcbb-eac93423ea7c None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:41.220067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1f9a3-4537-4ae1-8d44-4b312b108607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:41.220708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1f9a3-4537-4ae1-8d44-4b312b108607 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:41.220986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1f9a3-4537-4ae1-8d44-4b312b108607 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:41.221523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1f9a3-4537-4ae1-8d44-4b312b108607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:41.232441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1f9a3-4537-4ae1-8d44-4b312b108607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:41.234056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1f9a3-4537-4ae1-8d44-4b312b108607 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-b2a1f9a3-4537-4ae1-8d44-4b312b108607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:41.249660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:41.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:41.255779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:41.255779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:41.255779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:41.255779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:41.255779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 640/1278] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:41.257865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f3ec4-2273-45dd-9513-a766d4ad6489 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:41.367378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f3ec4-2273-45dd-9513-a766d4ad6489 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:41.404246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf54ca-5deb-4f65-bb36-f628dc986e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:41.404849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf54ca-5deb-4f65-bb36-f628dc986e2c None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:41.405131 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf54ca-5deb-4f65-bb36-f628dc986e2c None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:41.405516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf54ca-5deb-4f65-bb36-f628dc986e2c None admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:41.406123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf54ca-5deb-4f65-bb36-f628dc986e2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:41.406431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf54ca-5deb-4f65-bb36-f628dc986e2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:41.407784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf54ca-5deb-4f65-bb36-f628dc986e2c None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:41.419875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 639/1279] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:41.425687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1f9a3-4537-4ae1-8d44-4b312b108607 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:41.461898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:41.532662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1f9a3-4537-4ae1-8d44-4b312b108607 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:41.570113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b88e44-d7a7-4dab-b4ab-429277e958c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:41.570778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b88e44-d7a7-4dab-b4ab-429277e958c9 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:41.571048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b88e44-d7a7-4dab-b4ab-429277e958c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:41.571324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b88e44-d7a7-4dab-b4ab-429277e958c9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:41.579108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b88e44-d7a7-4dab-b4ab-429277e958c9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:41.580556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b88e44-d7a7-4dab-b4ab-429277e958c9 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-c9b88e44-d7a7-4dab-b4ab-429277e958c9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 21:25:41.594393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:41.596161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:41.598154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:41.598154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:41.598154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 21:25:41.598154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 21:25:41.598154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 21:25:41.598154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 21:25:41.598154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 21:25:41.598154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 21:25:41.598154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 21:25:41.598154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:41.598154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 641/1280] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:41.601744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf54ca-5deb-4f65-bb36-f628dc986e2c None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:41.606989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:41.706224 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf54ca-5deb-4f65-bb36-f628dc986e2c None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:41.742490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc202c-0410-47d1-a61d-898cfa3d86a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:41.743246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc202c-0410-47d1-a61d-898cfa3d86a7 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:41.743610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc202c-0410-47d1-a61d-898cfa3d86a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:41.744002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc202c-0410-47d1-a61d-898cfa3d86a7 None admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:41.744878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc202c-0410-47d1-a61d-898cfa3d86a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:41.745297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc202c-0410-47d1-a61d-898cfa3d86a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:41.747092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc202c-0410-47d1-a61d-898cfa3d86a7 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:41.766831 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 640/1281] 10.222.0.73 () {56 vars in 1143 bytes} [Thu Aug 4 21:25:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:41.772787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b88e44-d7a7-4dab-b4ab-429277e958c9 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:41.867164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:41.877403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b88e44-d7a7-4dab-b4ab-429277e958c9 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:41.913182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaf544-78ba-49cd-9a7d-5ced4ed74dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:41.913824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaf544-78ba-49cd-9a7d-5ced4ed74dda None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:41.914116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaf544-78ba-49cd-9a7d-5ced4ed74dda None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:41.914391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaf544-78ba-49cd-9a7d-5ced4ed74dda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:41.989580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaf544-78ba-49cd-9a7d-5ced4ed74dda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:41.991088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaf544-78ba-49cd-9a7d-5ced4ed74dda None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:42.082732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:42.092697 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 642/1282] 10.222.0.73 () {58 vars in 1282 bytes} [Thu Aug 4 21:25:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 21:25:42.099647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc202c-0410-47d1-a61d-898cfa3d86a7 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:42.165789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:42.206930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc202c-0410-47d1-a61d-898cfa3d86a7 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:42.244382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ec7aa-f08b-451d-b625-6365c5b06a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:42.245021 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ec7aa-f08b-451d-b625-6365c5b06a24 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:42.245305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ec7aa-f08b-451d-b625-6365c5b06a24 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:42.245580 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ec7aa-f08b-451d-b625-6365c5b06a24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:42.258490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ec7aa-f08b-451d-b625-6365c5b06a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:42.260120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ec7aa-f08b-451d-b625-6365c5b06a24 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-069ec7aa-f08b-451d-b625-6365c5b06a24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 21:25:42.276299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:42.277660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:42.278682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 21:25:42.278682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 21:25:42.278682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 21:25:42.278682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:42.278682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 641/1283] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:42.283267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaf544-78ba-49cd-9a7d-5ced4ed74dda None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:42.390334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaf544-78ba-49cd-9a7d-5ced4ed74dda None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:42.426839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350818e-e39e-49cd-9e6a-a10394038423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:42.427437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350818e-e39e-49cd-9e6a-a10394038423 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:42.427731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350818e-e39e-49cd-9e6a-a10394038423 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:42.427986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350818e-e39e-49cd-9e6a-a10394038423 None admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:42.428670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350818e-e39e-49cd-9e6a-a10394038423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:42.429010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350818e-e39e-49cd-9e6a-a10394038423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:42.430406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350818e-e39e-49cd-9e6a-a10394038423 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:42.442165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 643/1284] 10.222.0.73 () {56 vars in 1109 bytes} [Thu Aug 4 21:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:42.448016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ec7aa-f08b-451d-b625-6365c5b06a24 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:42.550930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ec7aa-f08b-451d-b625-6365c5b06a24 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:42.565160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:42.586923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911d827-acbd-447b-8041-901ad4223bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:42.587567 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911d827-acbd-447b-8041-901ad4223bbf None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:42.587852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911d827-acbd-447b-8041-901ad4223bbf None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:42.588129 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911d827-acbd-447b-8041-901ad4223bbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:42.594955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911d827-acbd-447b-8041-901ad4223bbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:42.596392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911d827-acbd-447b-8041-901ad4223bbf None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application [None req-c911d827-acbd-447b-8041-901ad4223bbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 21:25:42.608865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:42.610137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:42.611052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 21:25:42.611052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 21:25:42.611052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 21:25:42.611052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 21:25:42.611052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 21:25:42.611052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 21:25:42.611052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 21:25:42.611052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: ERROR keystone.server.flask.application Aug 04 21:25:42.611052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 642/1285] 10.222.0.73 () {56 vars in 1111 bytes} [Thu Aug 4 21:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:42.617000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350818e-e39e-49cd-9e6a-a10394038423 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:42.669800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:42.719984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350818e-e39e-49cd-9e6a-a10394038423 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:42.755765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2767c1-dc8c-4156-8cc2-4cdc763a3373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:42.756393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2767c1-dc8c-4156-8cc2-4cdc763a3373 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:42.756680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2767c1-dc8c-4156-8cc2-4cdc763a3373 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:42.757437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2767c1-dc8c-4156-8cc2-4cdc763a3373 None admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:42.758506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2767c1-dc8c-4156-8cc2-4cdc763a3373 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:42.758739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2767c1-dc8c-4156-8cc2-4cdc763a3373 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:42.760625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2767c1-dc8c-4156-8cc2-4cdc763a3373 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:42.774158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 644/1286] 10.222.0.73 () {56 vars in 1148 bytes} [Thu Aug 4 21:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:42.780557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911d827-acbd-447b-8041-901ad4223bbf None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:42.891804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911d827-acbd-447b-8041-901ad4223bbf None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:42.932558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd81bc2-0e02-4488-a81d-d6a3378ec2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:42.933185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd81bc2-0e02-4488-a81d-d6a3378ec2fa None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:42.933466 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd81bc2-0e02-4488-a81d-d6a3378ec2fa None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:42.933739 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd81bc2-0e02-4488-a81d-d6a3378ec2fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:42.934579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd81bc2-0e02-4488-a81d-d6a3378ec2fa None admin] RBAC: Adding query filter params (role.id=b39c7fb6d2e34182bdda29361f6c005e, scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1, user.id=fab36b186fa44ee18bf72228fc7ba758) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:42.934885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd81bc2-0e02-4488-a81d-d6a3378ec2fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:42.936322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd81bc2-0e02-4488-a81d-d6a3378ec2fa None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:42.942817 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 643/1287] 10.222.0.73 () {56 vars in 1374 bytes} [Thu Aug 4 21:25:42 2022] GET /identity/v3/role_assignments?user.id=fab36b186fa44ee18bf72228fc7ba758&scope.project.id=ca8a470d4b3c4a96abc633051a2f74f1&role.id=b39c7fb6d2e34182bdda29361f6c005e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:45.009621 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa010ff0-d285-4e42-b46e-588b2407fc4f None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.010205 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa010ff0-d285-4e42-b46e-588b2407fc4f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.010205 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa010ff0-d285-4e42-b46e-588b2407fc4f None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.011662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 645/1288] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:45.018054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244f440-00c6-4d07-9ce6-963ded6480a1 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.019308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244f440-00c6-4d07-9ce6-963ded6480a1 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.027780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244f440-00c6-4d07-9ce6-963ded6480a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.042765 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c244f440-00c6-4d07-9ce6-963ded6480a1 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:45.044306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c244f440-00c6-4d07-9ce6-963ded6480a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:45.083484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:45.128968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 644/1289] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:25:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:25:45.152874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa010ff0-d285-4e42-b46e-588b2407fc4f None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:45.296748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa010ff0-d285-4e42-b46e-588b2407fc4f None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:45.309750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:45.340107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c1fe9-1128-48b7-8196-abcd22f55b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:45.340793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c1fe9-1128-48b7-8196-abcd22f55b5d None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.341076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c1fe9-1128-48b7-8196-abcd22f55b5d None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.341346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c1fe9-1128-48b7-8196-abcd22f55b5d None admin] PATH_INFO: `/v3/services/placement` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.342085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c1fe9-1128-48b7-8196-abcd22f55b5d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:45.343377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c1fe9-1128-48b7-8196-abcd22f55b5d None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-ae9c1fe9-1128-48b7-8196-abcd22f55b5d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:45.353780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:45.355151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:25:45.356245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:45.360494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 646/1290] 10.222.0.73 () {56 vars in 1117 bytes} [Thu Aug 4 21:25:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:45.362273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244f440-00c6-4d07-9ce6-963ded6480a1 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:45.489317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244f440-00c6-4d07-9ce6-963ded6480a1 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:45.528834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb957a-04c2-4b1f-a9d6-957ebdc3c512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:45.529458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb957a-04c2-4b1f-a9d6-957ebdc3c512 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.529741 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb957a-04c2-4b1f-a9d6-957ebdc3c512 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.530027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb957a-04c2-4b1f-a9d6-957ebdc3c512 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.530729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb957a-04c2-4b1f-a9d6-957ebdc3c512 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:45.531036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb957a-04c2-4b1f-a9d6-957ebdc3c512 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:45.532410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb957a-04c2-4b1f-a9d6-957ebdc3c512 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:45.539479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 645/1291] 10.222.0.73 () {56 vars in 1116 bytes} [Thu Aug 4 21:25:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:45.545321 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c1fe9-1128-48b7-8196-abcd22f55b5d None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:45.654806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c1fe9-1128-48b7-8196-abcd22f55b5d None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:45.691604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c34dad-4b72-43fc-a4e8-09aabb93f98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:45.692214 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c34dad-4b72-43fc-a4e8-09aabb93f98c None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:45.692495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c34dad-4b72-43fc-a4e8-09aabb93f98c None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:45.692773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c34dad-4b72-43fc-a4e8-09aabb93f98c None admin] PATH_INFO: `/v3/services` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:45.693473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c34dad-4b72-43fc-a4e8-09aabb93f98c None admin] RBAC: Adding query filter params (type=placement) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:45.693782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c34dad-4b72-43fc-a4e8-09aabb93f98c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:45.695231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c34dad-4b72-43fc-a4e8-09aabb93f98c None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:45.699613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 647/1292] 10.222.0.73 () {56 vars in 1116 bytes} [Thu Aug 4 21:25:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:47.764742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d3778-c17a-4a32-9ffd-000e76ecce74 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:47.765297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d3778-c17a-4a32-9ffd-000e76ecce74 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:47.765297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d3778-c17a-4a32-9ffd-000e76ecce74 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:47.766728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 646/1293] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:47.773942 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bb39a-c3d4-4371-997b-6f86c015ec46 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:47.774760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bb39a-c3d4-4371-997b-6f86c015ec46 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:47.774760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bb39a-c3d4-4371-997b-6f86c015ec46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:47.795326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-040bb39a-c3d4-4371-997b-6f86c015ec46 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:47.796542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-040bb39a-c3d4-4371-997b-6f86c015ec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:47.831353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:47.869027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 648/1294] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:25:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:25:47.891727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d3778-c17a-4a32-9ffd-000e76ecce74 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:47.924355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:48.002553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d3778-c17a-4a32-9ffd-000e76ecce74 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:48.038570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010a208-9cc9-4bc6-a7b2-555de9723cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:48.039103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010a208-9cc9-4bc6-a7b2-555de9723cb8 None admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:48.039383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010a208-9cc9-4bc6-a7b2-555de9723cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:48.039656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010a208-9cc9-4bc6-a7b2-555de9723cb8 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:48.040345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010a208-9cc9-4bc6-a7b2-555de9723cb8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:48.041749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010a208-9cc9-4bc6-a7b2-555de9723cb8 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:48.149550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 647/1295] 10.222.0.73 () {60 vars in 1141 bytes} [Thu Aug 4 21:25:47 2022] POST /identity/v3/services => generated 246 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:25:50.283732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e23797-c7bc-4b5d-97f9-744b99f4a26a None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:50.284276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e23797-c7bc-4b5d-97f9-744b99f4a26a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:50.284496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e23797-c7bc-4b5d-97f9-744b99f4a26a None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:50.285946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 649/1296] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:50.291369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09db79-fd89-4dac-ba4d-89da22085fcb None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:50.293478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09db79-fd89-4dac-ba4d-89da22085fcb None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:50.293478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09db79-fd89-4dac-ba4d-89da22085fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:50.310248 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-fe09db79-fd89-4dac-ba4d-89da22085fcb None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:50.311521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-fe09db79-fd89-4dac-ba4d-89da22085fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:50.390499 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 648/1297] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:25:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 21:25:50.413897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e23797-c7bc-4b5d-97f9-744b99f4a26a None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:50.517727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e23797-c7bc-4b5d-97f9-744b99f4a26a None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:50.558520 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07aeec-9fb1-4b39-92c7-16eb249e7b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:50.559063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07aeec-9fb1-4b39-92c7-16eb249e7b02 None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:50.559342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07aeec-9fb1-4b39-92c7-16eb249e7b02 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:50.559613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07aeec-9fb1-4b39-92c7-16eb249e7b02 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:50.560239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07aeec-9fb1-4b39-92c7-16eb249e7b02 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:50.561668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07aeec-9fb1-4b39-92c7-16eb249e7b02 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-ce07aeec-9fb1-4b39-92c7-16eb249e7b02 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:50.573080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:50.575709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:50.577162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 650/1298] 10.222.0.73 () {56 vars in 1117 bytes} [Thu Aug 4 21:25:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 21:25:50.581549 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09db79-fd89-4dac-ba4d-89da22085fcb None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:50.606879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:50.697843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09db79-fd89-4dac-ba4d-89da22085fcb None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:50.739315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc5bff-ef79-4025-b7fc-b4f5d96eb0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:50.739952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc5bff-ef79-4025-b7fc-b4f5d96eb0d6 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:50.740234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc5bff-ef79-4025-b7fc-b4f5d96eb0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:50.740616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc5bff-ef79-4025-b7fc-b4f5d96eb0d6 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:50.741323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc5bff-ef79-4025-b7fc-b4f5d96eb0d6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:50.741631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc5bff-ef79-4025-b7fc-b4f5d96eb0d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:50.743103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc5bff-ef79-4025-b7fc-b4f5d96eb0d6 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:50.748912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 649/1299] 10.222.0.73 () {56 vars in 1116 bytes} [Thu Aug 4 21:25:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:50.756094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07aeec-9fb1-4b39-92c7-16eb249e7b02 None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:50.792965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:50.864319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07aeec-9fb1-4b39-92c7-16eb249e7b02 None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:50.898668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5346fa-5012-4a90-9486-bcc6bec0443f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:50.899278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5346fa-5012-4a90-9486-bcc6bec0443f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:50.899556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5346fa-5012-4a90-9486-bcc6bec0443f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:50.899825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5346fa-5012-4a90-9486-bcc6bec0443f None admin] PATH_INFO: `/v3/endpoints` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:50.900576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5346fa-5012-4a90-9486-bcc6bec0443f None admin] RBAC: Adding query filter params (interface=public, service_id=9a39816aa88a4d5283056e08c6a4faf2, region_id=RegionOne) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:50.900900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5346fa-5012-4a90-9486-bcc6bec0443f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:50.902283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5346fa-5012-4a90-9486-bcc6bec0443f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:50.907290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 651/1300] 10.222.0.73 () {56 vars in 1251 bytes} [Thu Aug 4 21:25:50 2022] GET /identity/v3/endpoints?service_id=9a39816aa88a4d5283056e08c6a4faf2&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:50.914336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc5bff-ef79-4025-b7fc-b4f5d96eb0d6 None admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:51.001920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:51.020930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc5bff-ef79-4025-b7fc-b4f5d96eb0d6 None admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:51.055885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b47af-2ee3-4412-baec-06fab325336b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:51.056512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b47af-2ee3-4412-baec-06fab325336b None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:51.056795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b47af-2ee3-4412-baec-06fab325336b None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:51.057070 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b47af-2ee3-4412-baec-06fab325336b None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:51.057725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b47af-2ee3-4412-baec-06fab325336b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:51.059125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b47af-2ee3-4412-baec-06fab325336b None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:51.067126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 650/1301] 10.222.0.73 () {56 vars in 1087 bytes} [Thu Aug 4 21:25:50 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:25:53.186556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc37d-116a-45a2-8318-49b79d44a5ca None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.187150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc37d-116a-45a2-8318-49b79d44a5ca None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.187150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc37d-116a-45a2-8318-49b79d44a5ca None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.188411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 652/1302] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:25:53.194025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d00c2-7916-476e-aa45-89e110ef6d68 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.194518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d00c2-7916-476e-aa45-89e110ef6d68 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.194622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d00c2-7916-476e-aa45-89e110ef6d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.214586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-a62d00c2-7916-476e-aa45-89e110ef6d68 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:25:53.215794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a62d00c2-7916-476e-aa45-89e110ef6d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:25:53.287191 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 651/1303] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:25:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 21:25:53.310784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dc37d-116a-45a2-8318-49b79d44a5ca None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:53.412533 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:53.419593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dc37d-116a-45a2-8318-49b79d44a5ca None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:53.454833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bca99a-d896-48bf-a97b-3654aac0149f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:53.455390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bca99a-d896-48bf-a97b-3654aac0149f None admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.456161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bca99a-d896-48bf-a97b-3654aac0149f None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.456434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bca99a-d896-48bf-a97b-3654aac0149f None admin] PATH_INFO: `/v3/services/placement` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.457064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bca99a-d896-48bf-a97b-3654aac0149f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:53.458479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bca99a-d896-48bf-a97b-3654aac0149f None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application [None req-49bca99a-d896-48bf-a97b-3654aac0149f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 21:25:53.468008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 21:25:53.469213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application with Lock( Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self._enter() Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application return self.creator() Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application created_value = creator( Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: ERROR keystone.server.flask.application Aug 04 21:25:53.480201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 653/1304] 10.222.0.73 () {56 vars in 1117 bytes} [Thu Aug 4 21:25:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 21:25:53.481370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d00c2-7916-476e-aa45-89e110ef6d68 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:53.560128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:25:53.584284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d00c2-7916-476e-aa45-89e110ef6d68 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:53.621319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b94d92-e8dc-4668-887f-421fef39b4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:53.621941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b94d92-e8dc-4668-887f-421fef39b4a8 None admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.622254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b94d92-e8dc-4668-887f-421fef39b4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.622530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b94d92-e8dc-4668-887f-421fef39b4a8 None admin] PATH_INFO: `/v3/services` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.623209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b94d92-e8dc-4668-887f-421fef39b4a8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:25:53.623521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b94d92-e8dc-4668-887f-421fef39b4a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:53.624884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b94d92-e8dc-4668-887f-421fef39b4a8 None admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:53.632732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 652/1305] 10.222.0.73 () {56 vars in 1116 bytes} [Thu Aug 4 21:25:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:25:53.638743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bca99a-d896-48bf-a97b-3654aac0149f None admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:25:53.742366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bca99a-d896-48bf-a97b-3654aac0149f None admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:25:53.776704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314da1f0-5171-45b5-813d-2a494ec3da24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:25:53.777324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314da1f0-5171-45b5-813d-2a494ec3da24 None admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:25:53.777604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314da1f0-5171-45b5-813d-2a494ec3da24 None admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:25:53.777878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314da1f0-5171-45b5-813d-2a494ec3da24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:25:53.778627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314da1f0-5171-45b5-813d-2a494ec3da24 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:25:53.779999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314da1f0-5171-45b5-813d-2a494ec3da24 None admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:25:53.879878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 654/1306] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:25:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:26:03.627108 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=30 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:26:03.875174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:07.328929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c5e28-67a4-464f-9e19-4d0e1534bf18 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:07.329540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c5e28-67a4-464f-9e19-4d0e1534bf18 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:07.329540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c5e28-67a4-464f-9e19-4d0e1534bf18 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:07.330848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 653/1307] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:07.337100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376daad9-529f-4d9a-b4fc-5a8600b53c4e None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:07.337608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376daad9-529f-4d9a-b4fc-5a8600b53c4e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:07.337913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376daad9-529f-4d9a-b4fc-5a8600b53c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:07.417753 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-376daad9-529f-4d9a-b4fc-5a8600b53c4e None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:07.419166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-376daad9-529f-4d9a-b4fc-5a8600b53c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:07.546531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 655/1308] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:07.592312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82070c75-a50f-429f-88c1-64c5a64dfcc7 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:07.592940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82070c75-a50f-429f-88c1-64c5a64dfcc7 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:07.593427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82070c75-a50f-429f-88c1-64c5a64dfcc7 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:07.595496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 654/1309] 10.222.0.73 () {54 vars in 965 bytes} [Thu Aug 4 21:32:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:07.605071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679215a8-c2e2-48b7-9ecd-6089fe1f618c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:07.605514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679215a8-c2e2-48b7-9ecd-6089fe1f618c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:07.605936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679215a8-c2e2-48b7-9ecd-6089fe1f618c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:07.755649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-679215a8-c2e2-48b7-9ecd-6089fe1f618c None None] MFA Rules not processed for user `c98761f81496456fb3b5a5cffd4ce763`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:07.758770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-679215a8-c2e2-48b7-9ecd-6089fe1f618c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:07.805030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:07.963548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 656/1310] 10.222.0.73 () {58 vars in 1062 bytes} [Thu Aug 4 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:07.967888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82070c75-a50f-429f-88c1-64c5a64dfcc7 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:07.969917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-82070c75-a50f-429f-88c1-64c5a64dfcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:08.197927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82070c75-a50f-429f-88c1-64c5a64dfcc7 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:08.370920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f67666-3c3b-4269-bf1f-75a2fda19a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:08.372192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f67666-3c3b-4269-bf1f-75a2fda19a43 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:08.372641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f67666-3c3b-4269-bf1f-75a2fda19a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:08.373078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f67666-3c3b-4269-bf1f-75a2fda19a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:08.379437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-27f67666-3c3b-4269-bf1f-75a2fda19a43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:08.510746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:08.634039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f67666-3c3b-4269-bf1f-75a2fda19a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:08.636821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f67666-3c3b-4269-bf1f-75a2fda19a43 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:08.677639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 655/1311] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:11.820382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e63ba3-2447-45d9-bce1-70c9087af178 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:11.821083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e63ba3-2447-45d9-bce1-70c9087af178 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:11.821083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e63ba3-2447-45d9-bce1-70c9087af178 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:11.822526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 657/1312] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:11.827896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952157af-b61f-489a-b204-b7bf47df80ef None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:11.828592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952157af-b61f-489a-b204-b7bf47df80ef None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:11.828592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952157af-b61f-489a-b204-b7bf47df80ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:11.923693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-952157af-b61f-489a-b204-b7bf47df80ef None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:11.924905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-952157af-b61f-489a-b204-b7bf47df80ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:12.166763 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 656/1313] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:12.207056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e63ba3-2447-45d9-bce1-70c9087af178 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:12.208629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b6e63ba3-2447-45d9-bce1-70c9087af178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:12.321081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e63ba3-2447-45d9-bce1-70c9087af178 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:12.396055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:12.398805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e0d70-6106-45d3-8e57-f56c90f0b103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:12.400096 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e0d70-6106-45d3-8e57-f56c90f0b103 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:12.400373 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e0d70-6106-45d3-8e57-f56c90f0b103 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:12.400643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e0d70-6106-45d3-8e57-f56c90f0b103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:12.402388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9d8e0d70-6106-45d3-8e57-f56c90f0b103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:12.518156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e0d70-6106-45d3-8e57-f56c90f0b103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:12.519559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e0d70-6106-45d3-8e57-f56c90f0b103 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:12.540621 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 658/1314] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:15.747913 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ff044-7068-4fe5-9799-14630f3681ec None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:15.748472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ff044-7068-4fe5-9799-14630f3681ec None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:15.748573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ff044-7068-4fe5-9799-14630f3681ec None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:15.750385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 657/1315] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:15.756653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83055702-1129-4a08-9a48-0496cc999f94 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:15.757148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83055702-1129-4a08-9a48-0496cc999f94 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:15.757277 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83055702-1129-4a08-9a48-0496cc999f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:15.837492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-83055702-1129-4a08-9a48-0496cc999f94 None None] MFA Rules not processed for user `b1c1f798af854a34b553bedc48b56c28`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:15.838597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-83055702-1129-4a08-9a48-0496cc999f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:15.899785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:15.945930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 659/1316] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:15 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:15.989013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ff044-7068-4fe5-9799-14630f3681ec None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:15.991252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-186ff044-7068-4fe5-9799-14630f3681ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:16.001818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:16.105750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ff044-7068-4fe5-9799-14630f3681ec None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:16.183791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb75fec-8361-4898-9e15-cc3ae8148d7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:16.184410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb75fec-8361-4898-9e15-cc3ae8148d7a service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:16.184688 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb75fec-8361-4898-9e15-cc3ae8148d7a service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:16.184957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb75fec-8361-4898-9e15-cc3ae8148d7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:16.186620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-eeb75fec-8361-4898-9e15-cc3ae8148d7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:16.252689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:16.310696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb75fec-8361-4898-9e15-cc3ae8148d7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:16.312134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb75fec-8361-4898-9e15-cc3ae8148d7a service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:16.331728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 658/1317] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:15 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:32:19.321658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70854ea6-85ce-4022-af98-1f26a418c3af None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:19.322253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70854ea6-85ce-4022-af98-1f26a418c3af None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:19.322253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70854ea6-85ce-4022-af98-1f26a418c3af None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:19.323691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 660/1318] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:19.329107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a1c50-634d-47bf-abfb-d02a9c630255 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:19.329548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a1c50-634d-47bf-abfb-d02a9c630255 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:19.329682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a1c50-634d-47bf-abfb-d02a9c630255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:19.399857 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-6f2a1c50-634d-47bf-abfb-d02a9c630255 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:19.401067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6f2a1c50-634d-47bf-abfb-d02a9c630255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:19.498536 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 659/1319] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:19.537981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70854ea6-85ce-4022-af98-1f26a418c3af None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:19.539581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-70854ea6-85ce-4022-af98-1f26a418c3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:19.645334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70854ea6-85ce-4022-af98-1f26a418c3af None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:19.719163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f96c04-681f-4f74-8ad6-031903d192b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:19.719885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f96c04-681f-4f74-8ad6-031903d192b3 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:19.720169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f96c04-681f-4f74-8ad6-031903d192b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:19.720441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f96c04-681f-4f74-8ad6-031903d192b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:19.722102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b6f96c04-681f-4f74-8ad6-031903d192b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:19.731058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:19.834718 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f96c04-681f-4f74-8ad6-031903d192b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:19.836148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f96c04-681f-4f74-8ad6-031903d192b3 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:19.855251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 661/1320] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:23.172335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebed458-7fff-4619-a1c8-4b4aa7074de2 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:23.172927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebed458-7fff-4619-a1c8-4b4aa7074de2 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:23.172927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebed458-7fff-4619-a1c8-4b4aa7074de2 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:23.174274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 660/1321] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:23.179776 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a058c-e1cf-4570-8cb7-c1075e39ea5b None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:23.180534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a058c-e1cf-4570-8cb7-c1075e39ea5b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:23.180757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a058c-e1cf-4570-8cb7-c1075e39ea5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:23.251767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-4e7a058c-e1cf-4570-8cb7-c1075e39ea5b None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:23.252976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4e7a058c-e1cf-4570-8cb7-c1075e39ea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:23.316598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:23.360390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 662/1322] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:23.399399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebed458-7fff-4619-a1c8-4b4aa7074de2 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:23.400973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5ebed458-7fff-4619-a1c8-4b4aa7074de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:23.422912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:23.520710 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebed458-7fff-4619-a1c8-4b4aa7074de2 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:23.613361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348bacb-cd56-4bc7-a886-e265eea8029d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:23.614032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348bacb-cd56-4bc7-a886-e265eea8029d service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:23.614320 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348bacb-cd56-4bc7-a886-e265eea8029d service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:23.614861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348bacb-cd56-4bc7-a886-e265eea8029d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:23.617025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4348bacb-cd56-4bc7-a886-e265eea8029d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:23.703487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:23.747290 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348bacb-cd56-4bc7-a886-e265eea8029d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:23.748772 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348bacb-cd56-4bc7-a886-e265eea8029d service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:23.770465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 661/1323] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:25.979115 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04ae7c-927a-49a6-8bb4-11d037396fa9 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:25.979719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04ae7c-927a-49a6-8bb4-11d037396fa9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:25.979719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04ae7c-927a-49a6-8bb4-11d037396fa9 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:25.981020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 663/1324] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:25.986736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea1900-bdb3-462d-aae5-398da46867c2 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:25.987448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea1900-bdb3-462d-aae5-398da46867c2 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:25.987759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea1900-bdb3-462d-aae5-398da46867c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.061463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-85ea1900-bdb3-462d-aae5-398da46867c2 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:26.062801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-85ea1900-bdb3-462d-aae5-398da46867c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.170957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 662/1325] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:32:26.242446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af72189-e865-495f-90f8-adc28aa09fbe None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:26.242766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af72189-e865-495f-90f8-adc28aa09fbe None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:26.243004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af72189-e865-495f-90f8-adc28aa09fbe None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.244331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 664/1326] 10.222.0.73 () {54 vars in 965 bytes} [Thu Aug 4 21:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:26.250649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9f264-58a1-49a5-9a6f-d360d8eabf0b None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:26.251011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9f264-58a1-49a5-9a6f-d360d8eabf0b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:26.251369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9f264-58a1-49a5-9a6f-d360d8eabf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.286994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:26.336367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-79e9f264-58a1-49a5-9a6f-d360d8eabf0b None None] MFA Rules not processed for user `c98761f81496456fb3b5a5cffd4ce763`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:26.337584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-79e9f264-58a1-49a5-9a6f-d360d8eabf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.419695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 663/1327] 10.222.0.73 () {58 vars in 1062 bytes} [Thu Aug 4 21:32:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:26.429910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af72189-e865-495f-90f8-adc28aa09fbe None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:26.431796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3af72189-e865-495f-90f8-adc28aa09fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.550373 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af72189-e865-495f-90f8-adc28aa09fbe None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:26.627701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43e42b-0f8b-408f-b934-efd26510775b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:26.628318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e42b-0f8b-408f-b934-efd26510775b service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:26.628592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e42b-0f8b-408f-b934-efd26510775b service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:26.628862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e42b-0f8b-408f-b934-efd26510775b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:26.631026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5f43e42b-0f8b-408f-b934-efd26510775b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:26.653878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:26.760911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43e42b-0f8b-408f-b934-efd26510775b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:26.762552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43e42b-0f8b-408f-b934-efd26510775b service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:26.782231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 665/1328] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:29.128309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ed1ca-e76f-481c-a667-f69ad4f70c13 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:29.128957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ed1ca-e76f-481c-a667-f69ad4f70c13 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:29.128957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ed1ca-e76f-481c-a667-f69ad4f70c13 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:29.131227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 664/1329] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:29.136278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9737059-7871-4745-89a1-d652e9ffea2b None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:29.136743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9737059-7871-4745-89a1-d652e9ffea2b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:29.137057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9737059-7871-4745-89a1-d652e9ffea2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:29.209828 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a9737059-7871-4745-89a1-d652e9ffea2b None None] MFA Rules not processed for user `b1c1f798af854a34b553bedc48b56c28`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:29.211007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a9737059-7871-4745-89a1-d652e9ffea2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:29.282779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:29.315988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 666/1330] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:29 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:29.355592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ed1ca-e76f-481c-a667-f69ad4f70c13 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:29.357133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-603ed1ca-e76f-481c-a667-f69ad4f70c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:29.492411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ed1ca-e76f-481c-a667-f69ad4f70c13 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:29.517347 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:29.587047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22682bb-897f-4e87-8e5c-1d9e1e3ead98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:29.587711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22682bb-897f-4e87-8e5c-1d9e1e3ead98 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:29.588026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22682bb-897f-4e87-8e5c-1d9e1e3ead98 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:29.588338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22682bb-897f-4e87-8e5c-1d9e1e3ead98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:29.589994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b22682bb-897f-4e87-8e5c-1d9e1e3ead98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:29.703278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22682bb-897f-4e87-8e5c-1d9e1e3ead98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:29.704830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22682bb-897f-4e87-8e5c-1d9e1e3ead98 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:29.725075 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 665/1331] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:32.601799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74914aaa-fa4e-46d0-954f-99769c9161ad None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:32.602564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74914aaa-fa4e-46d0-954f-99769c9161ad None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:32.602564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74914aaa-fa4e-46d0-954f-99769c9161ad None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:32.603869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 667/1332] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:32.609007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f479d-ea7e-427f-8817-adc80baf9af1 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:32.609450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f479d-ea7e-427f-8817-adc80baf9af1 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:32.609561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f479d-ea7e-427f-8817-adc80baf9af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:32.675034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:32.678177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f24f479d-ea7e-427f-8817-adc80baf9af1 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:32.679408 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f24f479d-ea7e-427f-8817-adc80baf9af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:32.776848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 666/1333] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:32.816099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74914aaa-fa4e-46d0-954f-99769c9161ad None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:32.818267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-74914aaa-fa4e-46d0-954f-99769c9161ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:32.923550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74914aaa-fa4e-46d0-954f-99769c9161ad None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:32.997361 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610457d5-09a8-46f5-a844-ec5b7a7a1e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:32.997971 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610457d5-09a8-46f5-a844-ec5b7a7a1e4f service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:32.998280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610457d5-09a8-46f5-a844-ec5b7a7a1e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:32.998545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610457d5-09a8-46f5-a844-ec5b7a7a1e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:33.000169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-610457d5-09a8-46f5-a844-ec5b7a7a1e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:33.027732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:33.109182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610457d5-09a8-46f5-a844-ec5b7a7a1e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:33.110640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610457d5-09a8-46f5-a844-ec5b7a7a1e4f service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:33.130664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 668/1334] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:35.229111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44d75-0af3-414f-9918-e1daac6b8a0c None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.229715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44d75-0af3-414f-9918-e1daac6b8a0c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.229715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44d75-0af3-414f-9918-e1daac6b8a0c None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.231036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 667/1335] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:35.236181 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39d4c4-79ea-4849-8379-11201ca5f738 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.236895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39d4c4-79ea-4849-8379-11201ca5f738 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.237028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39d4c4-79ea-4849-8379-11201ca5f738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.303931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-2b39d4c4-79ea-4849-8379-11201ca5f738 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:35.305143 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2b39d4c4-79ea-4849-8379-11201ca5f738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:35.381408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:35.404145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 669/1336] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:35.445757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44d75-0af3-414f-9918-e1daac6b8a0c None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:35.447426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d5f44d75-0af3-414f-9918-e1daac6b8a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:35.559430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44d75-0af3-414f-9918-e1daac6b8a0c None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:35.589947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:35.636989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec37aa-00a8-450c-bb9a-cd1eb858a9d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:35.637605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec37aa-00a8-450c-bb9a-cd1eb858a9d4 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:35.637888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec37aa-00a8-450c-bb9a-cd1eb858a9d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:35.638191 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec37aa-00a8-450c-bb9a-cd1eb858a9d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:35.639829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6bec37aa-00a8-450c-bb9a-cd1eb858a9d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:35.747602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec37aa-00a8-450c-bb9a-cd1eb858a9d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:35.749037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec37aa-00a8-450c-bb9a-cd1eb858a9d4 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:35.768169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 668/1337] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:38.704844 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee14ae-18be-416e-b09f-be2279dc7844 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:38.705394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee14ae-18be-416e-b09f-be2279dc7844 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:38.705394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee14ae-18be-416e-b09f-be2279dc7844 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:38.706781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 670/1338] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:38.712155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66b5a-c703-43b9-bb98-e8f1ec7c552d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:38.712588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66b5a-c703-43b9-bb98-e8f1ec7c552d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:38.712699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66b5a-c703-43b9-bb98-e8f1ec7c552d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:38.780661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f7a66b5a-c703-43b9-bb98-e8f1ec7c552d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:38.781796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f7a66b5a-c703-43b9-bb98-e8f1ec7c552d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:38.790091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:38.877195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 669/1339] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:38.914692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee14ae-18be-416e-b09f-be2279dc7844 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:38.916245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-72ee14ae-18be-416e-b09f-be2279dc7844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:39.019182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee14ae-18be-416e-b09f-be2279dc7844 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:39.091747 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90c983-5771-4039-a1b4-14f3bdfb4fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:39.092284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90c983-5771-4039-a1b4-14f3bdfb4fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:39.092567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90c983-5771-4039-a1b4-14f3bdfb4fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:39.092841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90c983-5771-4039-a1b4-14f3bdfb4fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:39.094519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ce90c983-5771-4039-a1b4-14f3bdfb4fa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:39.129811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:39.204726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90c983-5771-4039-a1b4-14f3bdfb4fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:39.206223 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90c983-5771-4039-a1b4-14f3bdfb4fa0 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:39.225373 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 671/1340] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:43.626597 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b36cc-8173-41ff-b606-7bab5ef3cd1c None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:43.626983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b36cc-8173-41ff-b606-7bab5ef3cd1c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:43.626983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b36cc-8173-41ff-b606-7bab5ef3cd1c None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:43.628292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 670/1341] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:43.633662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afde4f9-3dca-4796-93e4-91a71a447b8d None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:43.634163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afde4f9-3dca-4796-93e4-91a71a447b8d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:43.634264 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afde4f9-3dca-4796-93e4-91a71a447b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:43.698918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-5afde4f9-3dca-4796-93e4-91a71a447b8d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:43.700124 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5afde4f9-3dca-4796-93e4-91a71a447b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:43.786760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:43.798385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 672/1342] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:43.838361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b36cc-8173-41ff-b606-7bab5ef3cd1c None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:43.839954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b23b36cc-8173-41ff-b606-7bab5ef3cd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:43.945437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b36cc-8173-41ff-b606-7bab5ef3cd1c None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:43.983231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:44.021694 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cc6e5-acdc-4e63-8a3b-9f1636648788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:44.022342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cc6e5-acdc-4e63-8a3b-9f1636648788 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:44.022621 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cc6e5-acdc-4e63-8a3b-9f1636648788 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:44.022889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cc6e5-acdc-4e63-8a3b-9f1636648788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:44.024500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-fe1cc6e5-acdc-4e63-8a3b-9f1636648788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:44.132958 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cc6e5-acdc-4e63-8a3b-9f1636648788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:44.134506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cc6e5-acdc-4e63-8a3b-9f1636648788 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:44.153528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 671/1343] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:47.153941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec20bb7-4785-499c-b4f8-f7392f2a73d1 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:47.154522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec20bb7-4785-499c-b4f8-f7392f2a73d1 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:47.154522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec20bb7-4785-499c-b4f8-f7392f2a73d1 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:47.155911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 673/1344] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:47.161506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04716214-d50e-44d3-9c38-25b4c60b728b None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:47.161938 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04716214-d50e-44d3-9c38-25b4c60b728b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:47.162236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04716214-d50e-44d3-9c38-25b4c60b728b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:47.230735 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-04716214-d50e-44d3-9c38-25b4c60b728b None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:47.231845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-04716214-d50e-44d3-9c38-25b4c60b728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:47.251698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:47.339231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 672/1345] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:47.377985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec20bb7-4785-499c-b4f8-f7392f2a73d1 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:47.384075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3ec20bb7-4785-499c-b4f8-f7392f2a73d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:47.486075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec20bb7-4785-499c-b4f8-f7392f2a73d1 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:47.564809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d036a4-6d77-4099-a9e9-e5ffc7cacbf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:47.565426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d036a4-6d77-4099-a9e9-e5ffc7cacbf4 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:47.565707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d036a4-6d77-4099-a9e9-e5ffc7cacbf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:47.565982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d036a4-6d77-4099-a9e9-e5ffc7cacbf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:47.568257 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-56d036a4-6d77-4099-a9e9-e5ffc7cacbf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:47.610928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:47.683879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d036a4-6d77-4099-a9e9-e5ffc7cacbf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:47.685320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d036a4-6d77-4099-a9e9-e5ffc7cacbf4 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:47.703481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 674/1346] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:51.941345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb47d99-13ee-41bf-a977-6b505dfab718 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:51.941871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb47d99-13ee-41bf-a977-6b505dfab718 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:51.942447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb47d99-13ee-41bf-a977-6b505dfab718 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:51.945323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 673/1347] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:32:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:51.949385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a06c3-06e1-4615-a015-ea4d256c2ac4 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:51.949878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a06c3-06e1-4615-a015-ea4d256c2ac4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:51.949974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a06c3-06e1-4615-a015-ea4d256c2ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:52.028422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d27a06c3-06e1-4615-a015-ea4d256c2ac4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:52.029695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d27a06c3-06e1-4615-a015-ea4d256c2ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:52.146878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:52.150945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 675/1348] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:32:52.189462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb47d99-13ee-41bf-a977-6b505dfab718 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:32:52.191015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-dcb47d99-13ee-41bf-a977-6b505dfab718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:52.309612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb47d99-13ee-41bf-a977-6b505dfab718 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:32:52.367066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:52.395969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bb7bc-9898-43ca-a449-77a88fcdee63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:32:52.396672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bb7bc-9898-43ca-a449-77a88fcdee63 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:52.396882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bb7bc-9898-43ca-a449-77a88fcdee63 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:52.397159 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bb7bc-9898-43ca-a449-77a88fcdee63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:52.398875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7c8bb7bc-9898-43ca-a449-77a88fcdee63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:52.527886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bb7bc-9898-43ca-a449-77a88fcdee63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:32:52.529322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bb7bc-9898-43ca-a449-77a88fcdee63 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:32:52.550192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 674/1349] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:32:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:32:57.538114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e82d82-4e1f-4341-8da8-92adaa52936b None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:57.538634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e82d82-4e1f-4341-8da8-92adaa52936b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:57.538634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e82d82-4e1f-4341-8da8-92adaa52936b None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:57.539988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 676/1350] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:32:57.545261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ebd64-b9aa-4757-b850-c25be5ae33d3 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:32:57.545685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ebd64-b9aa-4757-b850-c25be5ae33d3 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:32:57.550696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ebd64-b9aa-4757-b850-c25be5ae33d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:32:57.657453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-a00ebd64-b9aa-4757-b850-c25be5ae33d3 None None] MFA Rules not processed for user `fab36b186fa44ee18bf72228fc7ba758`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:32:57.658853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a00ebd64-b9aa-4757-b850-c25be5ae33d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:32:57.693466 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:32:57.820361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 675/1351] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:32:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:01.979232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b68778-b5e3-41dd-a1a9-25aee1bf5a2a None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:01.980211 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b68778-b5e3-41dd-a1a9-25aee1bf5a2a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:01.980516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b68778-b5e3-41dd-a1a9-25aee1bf5a2a None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:01.982043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 677/1352] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:33:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:01.988322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87476217-881c-459e-ba51-dd7596066492 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:01.988812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87476217-881c-459e-ba51-dd7596066492 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:01.989133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87476217-881c-459e-ba51-dd7596066492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:02.088167 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-87476217-881c-459e-ba51-dd7596066492 None None] MFA Rules not processed for user `fab36b186fa44ee18bf72228fc7ba758`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:02.089717 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-87476217-881c-459e-ba51-dd7596066492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:02.166208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:02.202512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 676/1353] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:33:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:03.752977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843ba15-bd7e-4938-8c15-879d68d3a12d None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:03.754080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843ba15-bd7e-4938-8c15-879d68d3a12d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:03.754261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843ba15-bd7e-4938-8c15-879d68d3a12d None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:03.755635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 678/1354] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:33:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:03.760795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c2b6b-7640-456a-908a-787c0f158e29 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:03.761235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c2b6b-7640-456a-908a-787c0f158e29 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:03.761342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c2b6b-7640-456a-908a-787c0f158e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:03.836277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-9b9c2b6b-7640-456a-908a-787c0f158e29 None None] MFA Rules not processed for user `fab36b186fa44ee18bf72228fc7ba758`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:03.837488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-9b9c2b6b-7640-456a-908a-787c0f158e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:03.946268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 677/1355] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:33:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:07.379719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f4368-c39e-4eb7-a90f-d3a0bbe815b3 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:07.380278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f4368-c39e-4eb7-a90f-d3a0bbe815b3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:07.380278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f4368-c39e-4eb7-a90f-d3a0bbe815b3 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:07.381819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 679/1356] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:07.388063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830542aa-3920-408d-9cdc-fcde7e23a82f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:07.388947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830542aa-3920-408d-9cdc-fcde7e23a82f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:07.389446 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830542aa-3920-408d-9cdc-fcde7e23a82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:07.444369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:07.490325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-830542aa-3920-408d-9cdc-fcde7e23a82f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:07.492077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-830542aa-3920-408d-9cdc-fcde7e23a82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:07.600217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 678/1357] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:07.655528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2245d-36bc-48f8-bbb4-4e30262ebffa None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:07.656144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2245d-36bc-48f8-bbb4-4e30262ebffa None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:07.656594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2245d-36bc-48f8-bbb4-4e30262ebffa None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:07.658591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 680/1358] 10.222.0.73 () {54 vars in 962 bytes} [Thu Aug 4 21:33:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:07.666678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5222e91-98a0-4cdf-9c31-9d1ddd2ce93f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:07.666961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5222e91-98a0-4cdf-9c31-9d1ddd2ce93f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:07.667233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5222e91-98a0-4cdf-9c31-9d1ddd2ce93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:07.755442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-e5222e91-98a0-4cdf-9c31-9d1ddd2ce93f None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:07.757021 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e5222e91-98a0-4cdf-9c31-9d1ddd2ce93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:07.811886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:07.908722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 679/1359] 10.222.0.73 () {58 vars in 1059 bytes} [Thu Aug 4 21:33:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:07.917596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2245d-36bc-48f8-bbb4-4e30262ebffa None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:07.921256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3ed2245d-36bc-48f8-bbb4-4e30262ebffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:08.099031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2245d-36bc-48f8-bbb4-4e30262ebffa None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:08.176081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:08.218877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fce24-17cb-43ee-a679-5e16676a3f74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:08.219493 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fce24-17cb-43ee-a679-5e16676a3f74 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:08.219774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fce24-17cb-43ee-a679-5e16676a3f74 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:08.220052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fce24-17cb-43ee-a679-5e16676a3f74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:08.221826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-419fce24-17cb-43ee-a679-5e16676a3f74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:08.325321 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fce24-17cb-43ee-a679-5e16676a3f74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:08.326945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fce24-17cb-43ee-a679-5e16676a3f74 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:08.345976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 681/1360] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:33:10.603830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f3415-a4ba-4027-9cd3-f515b06d1698 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:10.604518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f3415-a4ba-4027-9cd3-f515b06d1698 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:10.604518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f3415-a4ba-4027-9cd3-f515b06d1698 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:10.609557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 680/1361] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:10.611634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f280bf4-4dd9-49d6-9912-f6537469b308 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:10.612246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f280bf4-4dd9-49d6-9912-f6537469b308 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:10.612806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f280bf4-4dd9-49d6-9912-f6537469b308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:10.682674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-5f280bf4-4dd9-49d6-9912-f6537469b308 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:10.684018 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5f280bf4-4dd9-49d6-9912-f6537469b308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:10.724073 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:10.786220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 682/1362] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:10.831839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f3415-a4ba-4027-9cd3-f515b06d1698 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:10.834382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ab2f3415-a4ba-4027-9cd3-f515b06d1698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:11.008350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f3415-a4ba-4027-9cd3-f515b06d1698 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:11.018957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:11.161254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecab9b-26d8-4707-80ef-d1a3cf5bb823 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:11.161811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecab9b-26d8-4707-80ef-d1a3cf5bb823 service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:11.162123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecab9b-26d8-4707-80ef-d1a3cf5bb823 service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:11.162412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecab9b-26d8-4707-80ef-d1a3cf5bb823 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:11.164001 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bdecab9b-26d8-4707-80ef-d1a3cf5bb823 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:11.265763 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecab9b-26d8-4707-80ef-d1a3cf5bb823 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:11.267925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecab9b-26d8-4707-80ef-d1a3cf5bb823 service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:11.289366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 681/1363] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:13.801266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a8c8f-deee-43b2-b2c4-7b0d2bec25ec None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:13.802348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a8c8f-deee-43b2-b2c4-7b0d2bec25ec None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:13.802840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a8c8f-deee-43b2-b2c4-7b0d2bec25ec None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:13.804402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 683/1364] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:13.809672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d4e88-6391-47e4-a8c4-744dc86efdc1 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:13.810169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d4e88-6391-47e4-a8c4-744dc86efdc1 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:13.810258 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d4e88-6391-47e4-a8c4-744dc86efdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:13.826747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:13.880227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-3d5d4e88-6391-47e4-a8c4-744dc86efdc1 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:13.881439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3d5d4e88-6391-47e4-a8c4-744dc86efdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:13.981708 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 682/1365] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:14.029969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038d8d3-b241-4631-989c-94cccd62e453 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:14.030377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038d8d3-b241-4631-989c-94cccd62e453 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:14.030732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038d8d3-b241-4631-989c-94cccd62e453 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:14.032142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 684/1366] 10.222.0.73 () {54 vars in 962 bytes} [Thu Aug 4 21:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:14.038143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b37cc-003f-4b85-a94a-e5d9b4c5ef77 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:14.038433 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b37cc-003f-4b85-a94a-e5d9b4c5ef77 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:14.038707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b37cc-003f-4b85-a94a-e5d9b4c5ef77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:14.119091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-935b37cc-003f-4b85-a94a-e5d9b4c5ef77 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:14.120288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-935b37cc-003f-4b85-a94a-e5d9b4c5ef77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:14.155998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:14.242417 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 683/1367] 10.222.0.73 () {58 vars in 1059 bytes} [Thu Aug 4 21:33:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:14.250797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038d8d3-b241-4631-989c-94cccd62e453 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:14.252793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1038d8d3-b241-4631-989c-94cccd62e453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:14.429755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038d8d3-b241-4631-989c-94cccd62e453 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:14.453931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:14.561908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66229d3c-4c77-4542-b59b-932e0361c6e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:14.562640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66229d3c-4c77-4542-b59b-932e0361c6e1 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:14.563028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66229d3c-4c77-4542-b59b-932e0361c6e1 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:14.563364 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66229d3c-4c77-4542-b59b-932e0361c6e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:14.565698 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-66229d3c-4c77-4542-b59b-932e0361c6e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:14.663267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66229d3c-4c77-4542-b59b-932e0361c6e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:14.665071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66229d3c-4c77-4542-b59b-932e0361c6e1 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:14.685007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 685/1368] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:15.649955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873b79b-1d45-4155-bea7-fa1a9373dfae None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:15.650553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873b79b-1d45-4155-bea7-fa1a9373dfae None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:15.650553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873b79b-1d45-4155-bea7-fa1a9373dfae None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:15.652070 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 684/1369] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:15.657203 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66274-f034-4479-b51e-fc2f3848f8cf None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:15.657627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66274-f034-4479-b51e-fc2f3848f8cf None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:15.657797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66274-f034-4479-b51e-fc2f3848f8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:15.694129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:15.730670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-16e66274-f034-4479-b51e-fc2f3848f8cf None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:15.731904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-16e66274-f034-4479-b51e-fc2f3848f8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:15.838199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 686/1370] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:16.213084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e36dc3-6916-4be6-92d7-aca4a95e79fd None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:16.213441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e36dc3-6916-4be6-92d7-aca4a95e79fd None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:16.213843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e36dc3-6916-4be6-92d7-aca4a95e79fd None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:16.215528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 685/1371] 10.222.0.73 () {54 vars in 964 bytes} [Thu Aug 4 21:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:16.220563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f75ea-49bf-4072-b3f3-c383dc219161 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:16.221198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f75ea-49bf-4072-b3f3-c383dc219161 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:16.221822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f75ea-49bf-4072-b3f3-c383dc219161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:16.304225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-058f75ea-49bf-4072-b3f3-c383dc219161 None None] MFA Rules not processed for user `f3264e17413e4c64b644c1b607cca15b`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:16.305653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-058f75ea-49bf-4072-b3f3-c383dc219161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:16.356753 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:16.391591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 687/1372] 10.222.0.73 () {58 vars in 1061 bytes} [Thu Aug 4 21:33:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:16.400336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e36dc3-6916-4be6-92d7-aca4a95e79fd None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:16.402072 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b2e36dc3-6916-4be6-92d7-aca4a95e79fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:16.516574 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e36dc3-6916-4be6-92d7-aca4a95e79fd None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:16.565004 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:16.606104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595753c-d372-4883-b257-fb64034c6337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:16.606727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595753c-d372-4883-b257-fb64034c6337 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:16.607004 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595753c-d372-4883-b257-fb64034c6337 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:16.607273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595753c-d372-4883-b257-fb64034c6337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:16.608878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7595753c-d372-4883-b257-fb64034c6337 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:16.731170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595753c-d372-4883-b257-fb64034c6337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:16.732588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595753c-d372-4883-b257-fb64034c6337 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:16.755942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 686/1373] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:16.910795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fca38-f8aa-44c5-a088-e0b455f27546 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:16.911247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fca38-f8aa-44c5-a088-e0b455f27546 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:16.911481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fca38-f8aa-44c5-a088-e0b455f27546 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:16.913004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 688/1374] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:16.918516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6aa1-ba73-4d2f-a89e-5d082eb7eada None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:16.918941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6aa1-ba73-4d2f-a89e-5d082eb7eada None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:16.919490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6aa1-ba73-4d2f-a89e-5d082eb7eada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:16.996258 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-b67b6aa1-ba73-4d2f-a89e-5d082eb7eada None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:16.997463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b67b6aa1-ba73-4d2f-a89e-5d082eb7eada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:17.020450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:17.114042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 687/1375] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:17.151395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fca38-f8aa-44c5-a088-e0b455f27546 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:17.152934 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-728fca38-f8aa-44c5-a088-e0b455f27546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:17.300524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fca38-f8aa-44c5-a088-e0b455f27546 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:17.353052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:17.415232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8a30f-6cc6-492c-b983-243d37291946 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:17.415844 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a30f-6cc6-492c-b983-243d37291946 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:17.416122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a30f-6cc6-492c-b983-243d37291946 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:17.416395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a30f-6cc6-492c-b983-243d37291946 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:17.418432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-00f8a30f-6cc6-492c-b983-243d37291946 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:17.510120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8a30f-6cc6-492c-b983-243d37291946 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:17.511712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8a30f-6cc6-492c-b983-243d37291946 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:17.529974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 689/1376] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:19.727746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e22e27-824f-4f25-a20f-cbd68d175c55 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:19.728256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e22e27-824f-4f25-a20f-cbd68d175c55 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:19.728350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e22e27-824f-4f25-a20f-cbd68d175c55 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:19.729691 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 688/1377] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:19.735062 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf89f47-6cff-4054-9fcf-e622def5aeeb None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:19.735447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf89f47-6cff-4054-9fcf-e622def5aeeb None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:19.735613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf89f47-6cff-4054-9fcf-e622def5aeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:19.807433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ebf89f47-6cff-4054-9fcf-e622def5aeeb None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:19.808630 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ebf89f47-6cff-4054-9fcf-e622def5aeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:19.815915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:19.916623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 690/1378] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:19.966032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e22e27-824f-4f25-a20f-cbd68d175c55 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:19.966032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-38e22e27-824f-4f25-a20f-cbd68d175c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:20.119962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e22e27-824f-4f25-a20f-cbd68d175c55 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:20.123650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:20.234967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d4b84-bf02-493d-b1ee-1a8fabf584dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:20.235587 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d4b84-bf02-493d-b1ee-1a8fabf584dc service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:20.235865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d4b84-bf02-493d-b1ee-1a8fabf584dc service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:20.236946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d4b84-bf02-493d-b1ee-1a8fabf584dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:20.238660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-386d4b84-bf02-493d-b1ee-1a8fabf584dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:20.338908 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d4b84-bf02-493d-b1ee-1a8fabf584dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:20.340501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d4b84-bf02-493d-b1ee-1a8fabf584dc service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:20.359383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 689/1379] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:22.655898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abefdc3-3958-41ba-9e44-f6cbc94b14b7 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:22.656427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abefdc3-3958-41ba-9e44-f6cbc94b14b7 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:22.656427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abefdc3-3958-41ba-9e44-f6cbc94b14b7 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:22.657941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 691/1380] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:22.663616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aced4d-8d67-42dc-bfe6-92a6813cd90e None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:22.664051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aced4d-8d67-42dc-bfe6-92a6813cd90e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:22.664224 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aced4d-8d67-42dc-bfe6-92a6813cd90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:22.675738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:22.734685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-15aced4d-8d67-42dc-bfe6-92a6813cd90e None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:22.735998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-15aced4d-8d67-42dc-bfe6-92a6813cd90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:22.848683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 690/1381] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:22.893259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abefdc3-3958-41ba-9e44-f6cbc94b14b7 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:22.894874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3abefdc3-3958-41ba-9e44-f6cbc94b14b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:23.036760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:23.046917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abefdc3-3958-41ba-9e44-f6cbc94b14b7 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:23.165207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8d416-f3ea-4c3d-ba19-fdc10a097f29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:23.165825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8d416-f3ea-4c3d-ba19-fdc10a097f29 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:23.166145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8d416-f3ea-4c3d-ba19-fdc10a097f29 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:23.166417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8d416-f3ea-4c3d-ba19-fdc10a097f29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:23.168057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-51a8d416-f3ea-4c3d-ba19-fdc10a097f29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:23.262883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8d416-f3ea-4c3d-ba19-fdc10a097f29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:23.264945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8d416-f3ea-4c3d-ba19-fdc10a097f29 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:23.278956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:23.291364 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 692/1382] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:25.608919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf87fcd-8855-4fa4-821a-f80a53772fdf None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:25.609453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf87fcd-8855-4fa4-821a-f80a53772fdf None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:25.609704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf87fcd-8855-4fa4-821a-f80a53772fdf None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:25.612145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 691/1383] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:25.617530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dab90-b948-42b9-83d7-115bb5025a8c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:25.617954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dab90-b948-42b9-83d7-115bb5025a8c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:25.618163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dab90-b948-42b9-83d7-115bb5025a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:25.687490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-239dab90-b948-42b9-83d7-115bb5025a8c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:25.688780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-239dab90-b948-42b9-83d7-115bb5025a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:25.789858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 693/1384] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:25.833164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf87fcd-8855-4fa4-821a-f80a53772fdf None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:25.834790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3cf87fcd-8855-4fa4-821a-f80a53772fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:25.928041 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:26.003355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf87fcd-8855-4fa4-821a-f80a53772fdf None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:26.144891 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb685b-4e70-41c3-be1a-c9043beeaac5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:26.145697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb685b-4e70-41c3-be1a-c9043beeaac5 service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:26.146337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb685b-4e70-41c3-be1a-c9043beeaac5 service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:26.146693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb685b-4e70-41c3-be1a-c9043beeaac5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:26.149168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-40bb685b-4e70-41c3-be1a-c9043beeaac5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:26.202371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:26.247805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb685b-4e70-41c3-be1a-c9043beeaac5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:26.249834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb685b-4e70-41c3-be1a-c9043beeaac5 service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:26.273104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 692/1385] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:28.330805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392541d-f1fd-4afd-9b2a-e42c35722cfd None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:28.331497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392541d-f1fd-4afd-9b2a-e42c35722cfd None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:28.331497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392541d-f1fd-4afd-9b2a-e42c35722cfd None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:28.332781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 694/1386] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:28.337959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5398419-2c75-45d0-8c5b-b98447aa856d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:28.338417 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5398419-2c75-45d0-8c5b-b98447aa856d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:28.338571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5398419-2c75-45d0-8c5b-b98447aa856d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:28.409745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-a5398419-2c75-45d0-8c5b-b98447aa856d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:28.411150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a5398419-2c75-45d0-8c5b-b98447aa856d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:28.511231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 693/1387] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:28.560398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392541d-f1fd-4afd-9b2a-e42c35722cfd None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:28.561955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4392541d-f1fd-4afd-9b2a-e42c35722cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:28.642873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:28.727756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392541d-f1fd-4afd-9b2a-e42c35722cfd None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:28.852172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c531437-5491-4df1-b401-e3c9590161ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:28.852895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531437-5491-4df1-b401-e3c9590161ba service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:28.853184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531437-5491-4df1-b401-e3c9590161ba service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:28.853458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531437-5491-4df1-b401-e3c9590161ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:28.855170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5c531437-5491-4df1-b401-e3c9590161ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:28.894929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:28.952779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c531437-5491-4df1-b401-e3c9590161ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:28.954576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c531437-5491-4df1-b401-e3c9590161ba service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:28.974668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 695/1388] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:31.171774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ca445-f09e-4936-8bb3-e267d8eea13a None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:31.172336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ca445-f09e-4936-8bb3-e267d8eea13a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:31.172336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ca445-f09e-4936-8bb3-e267d8eea13a None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:31.174773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 694/1389] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:31.180403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f50a1e-111a-4bf6-97f7-776b87cd6470 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:31.180900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f50a1e-111a-4bf6-97f7-776b87cd6470 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:31.181015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f50a1e-111a-4bf6-97f7-776b87cd6470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:31.245608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c2f50a1e-111a-4bf6-97f7-776b87cd6470 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:31.246828 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c2f50a1e-111a-4bf6-97f7-776b87cd6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:31.342031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 696/1390] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:31.388167 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ca445-f09e-4936-8bb3-e267d8eea13a None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:31.389692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-434ca445-f09e-4936-8bb3-e267d8eea13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:31.421959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:31.543319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ca445-f09e-4936-8bb3-e267d8eea13a None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:31.644915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:31.656416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff682-dee6-44f1-80eb-451345262a0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:31.657043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff682-dee6-44f1-80eb-451345262a0d service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:31.657325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff682-dee6-44f1-80eb-451345262a0d service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:31.658278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff682-dee6-44f1-80eb-451345262a0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:31.660125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-638ff682-dee6-44f1-80eb-451345262a0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:31.751105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ff682-dee6-44f1-80eb-451345262a0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:31.752719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ff682-dee6-44f1-80eb-451345262a0d service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:31.771749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 695/1391] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:32.423876 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440a970-1e94-4404-9d43-383f0523474f None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:32.424509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440a970-1e94-4404-9d43-383f0523474f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:32.424509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440a970-1e94-4404-9d43-383f0523474f None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:32.425678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 697/1392] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:32.430835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b236b0-009e-4ae4-b986-16ddaa00dba0 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:32.431284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b236b0-009e-4ae4-b986-16ddaa00dba0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:32.431392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b236b0-009e-4ae4-b986-16ddaa00dba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:32.496321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c4b236b0-009e-4ae4-b986-16ddaa00dba0 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:32.497500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c4b236b0-009e-4ae4-b986-16ddaa00dba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:32.558565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:32.602035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 696/1393] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:32.653318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477622c-a24a-4ffb-9057-c02fd04d71f0 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:32.653640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477622c-a24a-4ffb-9057-c02fd04d71f0 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:32.653947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477622c-a24a-4ffb-9057-c02fd04d71f0 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:32.655337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 698/1394] 10.222.0.73 () {54 vars in 966 bytes} [Thu Aug 4 21:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:32.660987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1159d73-2e5b-448e-bbc0-2d8b13beda37 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:32.661300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1159d73-2e5b-448e-bbc0-2d8b13beda37 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:32.661574 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1159d73-2e5b-448e-bbc0-2d8b13beda37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:32.738176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-d1159d73-2e5b-448e-bbc0-2d8b13beda37 None None] MFA Rules not processed for user `03a9a6bcfa1b469298774e09a8912b93`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:32.739376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d1159d73-2e5b-448e-bbc0-2d8b13beda37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:32.816894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 697/1395] 10.222.0.73 () {58 vars in 1063 bytes} [Thu Aug 4 21:33:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:32.824527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477622c-a24a-4ffb-9057-c02fd04d71f0 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:32.826188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d477622c-a24a-4ffb-9057-c02fd04d71f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:32.840052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:32.933083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477622c-a24a-4ffb-9057-c02fd04d71f0 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:33.006821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9314d-e061-4828-85ed-90ed3fc310b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a9a6bcfa1b469298774e09a8912b93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:33.007361 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9314d-e061-4828-85ed-90ed3fc310b5 service glance] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:33.007687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9314d-e061-4828-85ed-90ed3fc310b5 service glance] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:33.007968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9314d-e061-4828-85ed-90ed3fc310b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:33.009670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d5f9314d-e061-4828-85ed-90ed3fc310b5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:33.082199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:33.124620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9314d-e061-4828-85ed-90ed3fc310b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:33.126323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9314d-e061-4828-85ed-90ed3fc310b5 service glance] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:33.144781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 699/1396] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:33.934545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3ebc4-9204-4c62-ba9f-08574b7ea7bb None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:33.935097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3ebc4-9204-4c62-ba9f-08574b7ea7bb None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:33.935097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3ebc4-9204-4c62-ba9f-08574b7ea7bb None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:33.936441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 698/1397] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:33.942583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199136e-9c27-4005-991c-defca23bd855 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:33.943029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199136e-9c27-4005-991c-defca23bd855 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:33.943144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199136e-9c27-4005-991c-defca23bd855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.010739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-8199136e-9c27-4005-991c-defca23bd855 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:34.011970 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8199136e-9c27-4005-991c-defca23bd855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:34.084759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5a078-6179-468c-8ca3-91a44d30c719 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.085080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5a078-6179-468c-8ca3-91a44d30c719 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.085363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5a078-6179-468c-8ca3-91a44d30c719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.120952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 700/1398] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:34.152989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-87d5a078-6179-468c-8ca3-91a44d30c719 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:34.154133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-87d5a078-6179-468c-8ca3-91a44d30c719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:34.163251 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:34.166406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199136e-9c27-4005-991c-defca23bd855 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:34.167629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8199136e-9c27-4005-991c-defca23bd855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:34.195108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:34.244724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 699/1399] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:33:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:34.292722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199136e-9c27-4005-991c-defca23bd855 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:34.293799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bc992-a685-4070-b10b-aaf9a128df34 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.294124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bc992-a685-4070-b10b-aaf9a128df34 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.294407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bc992-a685-4070-b10b-aaf9a128df34 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.295990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 700/1400] 10.222.0.73 () {54 vars in 943 bytes} [Thu Aug 4 21:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:33:34.302257 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf626737-ddab-423c-8c44-91defec38c6a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.302547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf626737-ddab-423c-8c44-91defec38c6a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.302826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf626737-ddab-423c-8c44-91defec38c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.381654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bf626737-ddab-423c-8c44-91defec38c6a None None] MFA Rules not processed for user `3342e1039131496ba121b04685ec7b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:34.382917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bf626737-ddab-423c-8c44-91defec38c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:34.412180 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4265a-72fa-4f8b-9542-21d5b1cfe56b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:34.412794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4265a-72fa-4f8b-9542-21d5b1cfe56b service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.413073 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4265a-72fa-4f8b-9542-21d5b1cfe56b service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.413371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4265a-72fa-4f8b-9542-21d5b1cfe56b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.415063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e4e4265a-72fa-4f8b-9542-21d5b1cfe56b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:34.437317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:34.493441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:34.521181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 701/1401] 10.222.0.73 () {58 vars in 1040 bytes} [Thu Aug 4 21:33:34 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:34.529811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf626737-ddab-423c-8c44-91defec38c6a None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:34.540094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf626737-ddab-423c-8c44-91defec38c6a None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:34.542981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4265a-72fa-4f8b-9542-21d5b1cfe56b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:34.544668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4265a-72fa-4f8b-9542-21d5b1cfe56b service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:34.578191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 701/1402] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:33:34.661312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbfb9b-f05f-427a-a747-ad5a261e5add service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:34.664064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbfb9b-f05f-427a-a747-ad5a261e5add service swift] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:34.664064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbfb9b-f05f-427a-a747-ad5a261e5add service swift] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:34.664064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbfb9b-f05f-427a-a747-ad5a261e5add service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:34.664345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5fdbfb9b-f05f-427a-a747-ad5a261e5add service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:34.747068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbfb9b-f05f-427a-a747-ad5a261e5add service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:34.748529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbfb9b-f05f-427a-a747-ad5a261e5add service swift] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:34.756703 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 702/1403] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:33:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:33:36.835560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091defe7-0f42-488c-bf08-f451041ea02f None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:36.836085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091defe7-0f42-488c-bf08-f451041ea02f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:36.836085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091defe7-0f42-488c-bf08-f451041ea02f None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:36.837889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 702/1404] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:36.843317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8bd06-287e-4777-a38a-4fe246715357 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:36.843791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8bd06-287e-4777-a38a-4fe246715357 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:36.843875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8bd06-287e-4777-a38a-4fe246715357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:36.856140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:36.916449 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-87a8bd06-287e-4777-a38a-4fe246715357 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:36.917694 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-87a8bd06-287e-4777-a38a-4fe246715357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:37.028223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 703/1405] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:33:37.076106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091defe7-0f42-488c-bf08-f451041ea02f None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:37.077641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-091defe7-0f42-488c-bf08-f451041ea02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:37.251888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:37.254991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091defe7-0f42-488c-bf08-f451041ea02f None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:37.389828 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb0d4a-5f58-411e-a716-5bf3ad630e41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:37.392823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb0d4a-5f58-411e-a716-5bf3ad630e41 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:37.393338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb0d4a-5f58-411e-a716-5bf3ad630e41 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:37.393761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb0d4a-5f58-411e-a716-5bf3ad630e41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:37.395777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e9fb0d4a-5f58-411e-a716-5bf3ad630e41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:37.504329 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb0d4a-5f58-411e-a716-5bf3ad630e41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:37.504329 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb0d4a-5f58-411e-a716-5bf3ad630e41 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:37.535139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 703/1406] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:33:40.730848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c60ce-7f45-404d-ad26-80c7157f0d0d None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:40.731471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c60ce-7f45-404d-ad26-80c7157f0d0d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:40.731471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c60ce-7f45-404d-ad26-80c7157f0d0d None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:40.732746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 704/1407] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:40.739348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe313a86-ab54-4bbd-a7da-934562363de9 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:40.739910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe313a86-ab54-4bbd-a7da-934562363de9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:40.739910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe313a86-ab54-4bbd-a7da-934562363de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:40.744483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:40.815434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-fe313a86-ab54-4bbd-a7da-934562363de9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:40.816593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-fe313a86-ab54-4bbd-a7da-934562363de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:40.926268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 704/1408] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:40.970125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c60ce-7f45-404d-ad26-80c7157f0d0d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:33:40.971652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-323c60ce-7f45-404d-ad26-80c7157f0d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:41.067808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:41.139431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c60ce-7f45-404d-ad26-80c7157f0d0d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:33:41.263878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd3bfc-8b15-453a-b122-19703d487fcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:33:41.264502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd3bfc-8b15-453a-b122-19703d487fcf service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:41.264780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd3bfc-8b15-453a-b122-19703d487fcf service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:41.265055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd3bfc-8b15-453a-b122-19703d487fcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:41.267388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-67dd3bfc-8b15-453a-b122-19703d487fcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:41.325645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:41.371476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd3bfc-8b15-453a-b122-19703d487fcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:33:41.373068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd3bfc-8b15-453a-b122-19703d487fcf service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:33:41.392564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 705/1409] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:33:50.924816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=64 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:51.389307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=23 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:58.606727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879794b1-74cd-4755-a574-39bdab8b63f1 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:58.607286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879794b1-74cd-4755-a574-39bdab8b63f1 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:58.607286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879794b1-74cd-4755-a574-39bdab8b63f1 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:58.608622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 705/1410] 10.222.0.73 () {54 vars in 918 bytes} [Thu Aug 4 21:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:58.613858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f36c5-c5a0-4c5d-b064-f9ccf1d1bc44 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:58.614341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f36c5-c5a0-4c5d-b064-f9ccf1d1bc44 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:58.614480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f36c5-c5a0-4c5d-b064-f9ccf1d1bc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:58.689617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-dc9f36c5-c5a0-4c5d-b064-f9ccf1d1bc44 None None] MFA Rules not processed for user `fab36b186fa44ee18bf72228fc7ba758`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:58.690863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-dc9f36c5-c5a0-4c5d-b064-f9ccf1d1bc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:58.808294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 706/1411] 10.222.0.73 () {58 vars in 1015 bytes} [Thu Aug 4 21:33:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:33:58.823317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00740870-f717-40bb-a0ec-1783fc36f835 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:58.823623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00740870-f717-40bb-a0ec-1783fc36f835 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:58.823936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00740870-f717-40bb-a0ec-1783fc36f835 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:58.825261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 706/1412] 10.222.0.73 () {54 vars in 918 bytes} [Thu Aug 4 21:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:33:58.830523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ef02f-791e-4ac9-99b1-2e3b863acb0c None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:33:58.830813 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ef02f-791e-4ac9-99b1-2e3b863acb0c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:33:58.831094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ef02f-791e-4ac9-99b1-2e3b863acb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:33:58.903517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:33:58.906037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c68ef02f-791e-4ac9-99b1-2e3b863acb0c None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:33:58.907249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c68ef02f-791e-4ac9-99b1-2e3b863acb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:33:59.020900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 707/1413] 10.222.0.73 () {58 vars in 1015 bytes} [Thu Aug 4 21:33:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:01.492797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e899e5-649f-4876-b9a2-64a99905a301 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.493343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e899e5-649f-4876-b9a2-64a99905a301 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.493343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e899e5-649f-4876-b9a2-64a99905a301 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.494685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:01.499805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0385a-1ff2-49b0-a014-4c35565760b9 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.500249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0385a-1ff2-49b0-a014-4c35565760b9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.500356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0385a-1ff2-49b0-a014-4c35565760b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.566873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-efd0385a-1ff2-49b0-a014-4c35565760b9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:01.568072 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-efd0385a-1ff2-49b0-a014-4c35565760b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:01.621875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:01.665023 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:01.707833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e899e5-649f-4876-b9a2-64a99905a301 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:01.709367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-05e899e5-649f-4876-b9a2-64a99905a301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:01.853209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e899e5-649f-4876-b9a2-64a99905a301 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:01.927549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:01.967606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d5211-3004-42bb-ab05-855212b698e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:01.968224 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d5211-3004-42bb-ab05-855212b698e4 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:01.968509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d5211-3004-42bb-ab05-855212b698e4 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:01.968814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d5211-3004-42bb-ab05-855212b698e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:01.970518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8f3d5211-3004-42bb-ab05-855212b698e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:02.060850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d5211-3004-42bb-ab05-855212b698e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:02.062511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d5211-3004-42bb-ab05-855212b698e4 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:02.081051 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 708/1416] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:02.991042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f8c61-b602-48fa-91fa-88a97b3c8877 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:02.991607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f8c61-b602-48fa-91fa-88a97b3c8877 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:02.991607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f8c61-b602-48fa-91fa-88a97b3c8877 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:02.992954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:02.997823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aef6d-a4f9-407f-b5d7-7c09fb065541 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:02.998256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aef6d-a4f9-407f-b5d7-7c09fb065541 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:02.998405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aef6d-a4f9-407f-b5d7-7c09fb065541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:03.073769 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-9c8aef6d-a4f9-407f-b5d7-7c09fb065541 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:03.075026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9c8aef6d-a4f9-407f-b5d7-7c09fb065541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:03.102987 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:03.173879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:03.323787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f8c61-b602-48fa-91fa-88a97b3c8877 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:03.325366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-131f8c61-b602-48fa-91fa-88a97b3c8877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:03.472359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f8c61-b602-48fa-91fa-88a97b3c8877 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:03.513403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:03.588715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106b18b-40b1-4efc-b0d1-97e5fbcc3abb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:03.589252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b18b-40b1-4efc-b0d1-97e5fbcc3abb service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:03.589559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b18b-40b1-4efc-b0d1-97e5fbcc3abb service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:03.590117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b18b-40b1-4efc-b0d1-97e5fbcc3abb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:03.592419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a106b18b-40b1-4efc-b0d1-97e5fbcc3abb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:03.680529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106b18b-40b1-4efc-b0d1-97e5fbcc3abb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:03.682158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106b18b-40b1-4efc-b0d1-97e5fbcc3abb service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:03.700686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 710/1419] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:04.585618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450641f-f618-4826-8ba6-ae276aef82b2 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:04.586160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450641f-f618-4826-8ba6-ae276aef82b2 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:04.586252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450641f-f618-4826-8ba6-ae276aef82b2 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:04.587533 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:04.592526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ff766-59d9-4be4-8f19-c506fdecde0a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:04.592938 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ff766-59d9-4be4-8f19-c506fdecde0a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:04.593078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ff766-59d9-4be4-8f19-c506fdecde0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:04.647846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:04.669073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-978ff766-59d9-4be4-8f19-c506fdecde0a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:04.670319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-978ff766-59d9-4be4-8f19-c506fdecde0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:04.768476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:04.919914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450641f-f618-4826-8ba6-ae276aef82b2 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:04.921428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a450641f-f618-4826-8ba6-ae276aef82b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:05.065462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450641f-f618-4826-8ba6-ae276aef82b2 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:05.078119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:05.178386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49c205-b8cb-4a39-8ae6-c7a56a0c4709 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:05.179011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49c205-b8cb-4a39-8ae6-c7a56a0c4709 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:05.179293 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49c205-b8cb-4a39-8ae6-c7a56a0c4709 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:05.179566 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49c205-b8cb-4a39-8ae6-c7a56a0c4709 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:05.181199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3b49c205-b8cb-4a39-8ae6-c7a56a0c4709 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:05.269384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49c205-b8cb-4a39-8ae6-c7a56a0c4709 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:05.271019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49c205-b8cb-4a39-8ae6-c7a56a0c4709 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:05.289742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 711/1422] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:06.161744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d644a8-fdc6-4327-af2e-222eab3ccc44 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:06.162304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d644a8-fdc6-4327-af2e-222eab3ccc44 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:06.162304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d644a8-fdc6-4327-af2e-222eab3ccc44 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:06.163718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:34:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:06.168988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aad3bc-3fdf-45df-bd41-bf3c73471b00 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:06.169414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aad3bc-3fdf-45df-bd41-bf3c73471b00 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:06.169544 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aad3bc-3fdf-45df-bd41-bf3c73471b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:06.189298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:06.244605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-b0aad3bc-3fdf-45df-bd41-bf3c73471b00 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:06.245808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b0aad3bc-3fdf-45df-bd41-bf3c73471b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:06.342560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:06.492901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d644a8-fdc6-4327-af2e-222eab3ccc44 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:06.497366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-08d644a8-fdc6-4327-af2e-222eab3ccc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:06.605397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:06.641329 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d644a8-fdc6-4327-af2e-222eab3ccc44 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:06.761487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1332a-5bbe-4176-ae4c-d01fbb60545e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:06.762120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1332a-5bbe-4176-ae4c-d01fbb60545e service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:06.762400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1332a-5bbe-4176-ae4c-d01fbb60545e service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:06.762673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1332a-5bbe-4176-ae4c-d01fbb60545e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:06.764278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7ca1332a-5bbe-4176-ae4c-d01fbb60545e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:06.839798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:06.853628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1332a-5bbe-4176-ae4c-d01fbb60545e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:06.859880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1332a-5bbe-4176-ae4c-d01fbb60545e service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:06.875010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 713/1425] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:07.737159 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfc582-275c-41e7-ab55-5d76b301d64b None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:07.738540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfc582-275c-41e7-ab55-5d76b301d64b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:07.738725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfc582-275c-41e7-ab55-5d76b301d64b None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:07.740082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:07.745291 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7631441-a1a0-4b24-aa8d-465ab0e72705 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:07.745712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7631441-a1a0-4b24-aa8d-465ab0e72705 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:07.745843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7631441-a1a0-4b24-aa8d-465ab0e72705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:07.828976 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-d7631441-a1a0-4b24-aa8d-465ab0e72705 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:07.830228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d7631441-a1a0-4b24-aa8d-465ab0e72705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:07.928230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:08.081490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfc582-275c-41e7-ab55-5d76b301d64b None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:08.083504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-dbcfc582-275c-41e7-ab55-5d76b301d64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:08.171574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:08.232824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfc582-275c-41e7-ab55-5d76b301d64b None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:08.342926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dede5-d3d7-4e53-9ed5-59813f40b529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:08.343456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dede5-d3d7-4e53-9ed5-59813f40b529 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:08.343735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dede5-d3d7-4e53-9ed5-59813f40b529 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:08.344007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dede5-d3d7-4e53-9ed5-59813f40b529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:08.345622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-236dede5-d3d7-4e53-9ed5-59813f40b529 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:08.393668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:08.435643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dede5-d3d7-4e53-9ed5-59813f40b529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:08.437250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dede5-d3d7-4e53-9ed5-59813f40b529 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:08.455723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 714/1428] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:09.345419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d84073-8f76-466e-bafb-e6914117b754 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:09.346187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d84073-8f76-466e-bafb-e6914117b754 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:09.346187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d84073-8f76-466e-bafb-e6914117b754 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:09.348362 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:09.353870 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4124e-68d2-46f5-9f83-455cb328c929 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:09.354388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4124e-68d2-46f5-9f83-455cb328c929 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:09.354495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4124e-68d2-46f5-9f83-455cb328c929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:09.432820 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-4fc4124e-68d2-46f5-9f83-455cb328c929 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:09.434099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4fc4124e-68d2-46f5-9f83-455cb328c929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:09.533367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:09.685778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d84073-8f76-466e-bafb-e6914117b754 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:09.687360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-33d84073-8f76-466e-bafb-e6914117b754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:09.737302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:09.834677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d84073-8f76-466e-bafb-e6914117b754 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:09.949015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaca773-2d23-432f-bbce-d166d89d4b9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:09.949557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaca773-2d23-432f-bbce-d166d89d4b9a service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:09.949840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaca773-2d23-432f-bbce-d166d89d4b9a service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:09.950148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaca773-2d23-432f-bbce-d166d89d4b9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:09.951787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-9eaca773-2d23-432f-bbce-d166d89d4b9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:09.959098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:10.042784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaca773-2d23-432f-bbce-d166d89d4b9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:10.044501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaca773-2d23-432f-bbce-d166d89d4b9a service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:10.063178 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 716/1431] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:10.954476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8bb93-4c38-41d8-8ec9-7c19522af7ff None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:10.955070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8bb93-4c38-41d8-8ec9-7c19522af7ff None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:10.955070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8bb93-4c38-41d8-8ec9-7c19522af7ff None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:10.956366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:10.961747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333839b-e09f-44ca-8648-5acbba10c87a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:10.962294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333839b-e09f-44ca-8648-5acbba10c87a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:10.962416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333839b-e09f-44ca-8648-5acbba10c87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:11.044337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f333839b-e09f-44ca-8648-5acbba10c87a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:11.045561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f333839b-e09f-44ca-8648-5acbba10c87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:11.115950 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:11.145277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:11.295861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8bb93-4c38-41d8-8ec9-7c19522af7ff None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:11.297328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b3f8bb93-4c38-41d8-8ec9-7c19522af7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:11.314783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:11.441677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8bb93-4c38-41d8-8ec9-7c19522af7ff None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:11.529307 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:11.556491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310cc98-8c68-47f8-b21d-cfadfd404e58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:11.557052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310cc98-8c68-47f8-b21d-cfadfd404e58 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:11.557331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310cc98-8c68-47f8-b21d-cfadfd404e58 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:11.557602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310cc98-8c68-47f8-b21d-cfadfd404e58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:11.559296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5310cc98-8c68-47f8-b21d-cfadfd404e58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:11.647412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310cc98-8c68-47f8-b21d-cfadfd404e58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:11.649001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310cc98-8c68-47f8-b21d-cfadfd404e58 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:11.667304 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 717/1434] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:12.562211 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec22012-0372-48ae-bc25-eaa1e46f8df0 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:12.562756 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec22012-0372-48ae-bc25-eaa1e46f8df0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:12.562756 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec22012-0372-48ae-bc25-eaa1e46f8df0 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:12.564110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:12.569144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4ae46-873e-4827-8fb6-54dfdd13c695 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:12.569615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4ae46-873e-4827-8fb6-54dfdd13c695 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:12.569725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4ae46-873e-4827-8fb6-54dfdd13c695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:12.645704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c8b4ae46-873e-4827-8fb6-54dfdd13c695 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:12.646926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c8b4ae46-873e-4827-8fb6-54dfdd13c695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:12.688587 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:12.743264 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:12.894132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec22012-0372-48ae-bc25-eaa1e46f8df0 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:12.895959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5ec22012-0372-48ae-bc25-eaa1e46f8df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:13.041003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec22012-0372-48ae-bc25-eaa1e46f8df0 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:13.093791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:13.155923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f7150-4ed1-47a6-87a2-6ffe75a2d341 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:13.156449 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f7150-4ed1-47a6-87a2-6ffe75a2d341 service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:13.156732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f7150-4ed1-47a6-87a2-6ffe75a2d341 service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:13.157006 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f7150-4ed1-47a6-87a2-6ffe75a2d341 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:13.158661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e83f7150-4ed1-47a6-87a2-6ffe75a2d341 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:13.249011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f7150-4ed1-47a6-87a2-6ffe75a2d341 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:13.250647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f7150-4ed1-47a6-87a2-6ffe75a2d341 service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:13.268278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 719/1437] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:14.150916 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5bc8-ca89-4892-9ba1-6a4a5314fbe9 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:14.151474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5bc8-ca89-4892-9ba1-6a4a5314fbe9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:14.151474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5bc8-ca89-4892-9ba1-6a4a5314fbe9 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:14.152814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:14.157728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a20be-d414-4b2d-84a5-44dc910b3592 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:14.158160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a20be-d414-4b2d-84a5-44dc910b3592 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:14.158302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a20be-d414-4b2d-84a5-44dc910b3592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:14.233457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bb6a20be-d414-4b2d-84a5-44dc910b3592 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:14.234675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bb6a20be-d414-4b2d-84a5-44dc910b3592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:14.236926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:14.331450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:14.487836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b5bc8-ca89-4892-9ba1-6a4a5314fbe9 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:14.489582 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-311b5bc8-ca89-4892-9ba1-6a4a5314fbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:14.636806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b5bc8-ca89-4892-9ba1-6a4a5314fbe9 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:14.663929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:14.748912 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917a4c1-a3c9-4a97-9716-478a0c16ead1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:14.749437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a4c1-a3c9-4a97-9716-478a0c16ead1 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:14.749715 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a4c1-a3c9-4a97-9716-478a0c16ead1 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:14.749985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a4c1-a3c9-4a97-9716-478a0c16ead1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:14.751627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4917a4c1-a3c9-4a97-9716-478a0c16ead1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:14.839606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917a4c1-a3c9-4a97-9716-478a0c16ead1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:14.841200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917a4c1-a3c9-4a97-9716-478a0c16ead1 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:14.859030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 720/1440] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:16.770858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92754cb0-7fa2-4d3c-9dff-18296c1b1f0c None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:16.771430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92754cb0-7fa2-4d3c-9dff-18296c1b1f0c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:16.771430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92754cb0-7fa2-4d3c-9dff-18296c1b1f0c None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:16.772768 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:16.777940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc30d-e634-42b2-9dbe-bf378e01e9fd None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:16.778398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc30d-e634-42b2-9dbe-bf378e01e9fd None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:16.778521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc30d-e634-42b2-9dbe-bf378e01e9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:16.819394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:16.844211 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-bd6cc30d-e634-42b2-9dbe-bf378e01e9fd None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:16.845514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bd6cc30d-e634-42b2-9dbe-bf378e01e9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:16.942854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:16.981633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92754cb0-7fa2-4d3c-9dff-18296c1b1f0c None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:16.983242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-92754cb0-7fa2-4d3c-9dff-18296c1b1f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:17.089042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92754cb0-7fa2-4d3c-9dff-18296c1b1f0c None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:17.121230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:17.163450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6a1b8-2718-4205-bc47-0f9cb8eea522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:17.164065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6a1b8-2718-4205-bc47-0f9cb8eea522 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:17.164342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6a1b8-2718-4205-bc47-0f9cb8eea522 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:17.164612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6a1b8-2718-4205-bc47-0f9cb8eea522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:17.166298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-26d6a1b8-2718-4205-bc47-0f9cb8eea522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:17.276719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6a1b8-2718-4205-bc47-0f9cb8eea522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:17.278169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6a1b8-2718-4205-bc47-0f9cb8eea522 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:17.296160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 722/1443] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:19.450458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed7ac-b16a-47b2-9758-1385c1782265 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:19.451025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed7ac-b16a-47b2-9758-1385c1782265 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:19.451025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed7ac-b16a-47b2-9758-1385c1782265 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:19.452382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:19.457816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d321025-4861-4cbb-8690-2bbe10e1c329 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:19.458255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d321025-4861-4cbb-8690-2bbe10e1c329 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:19.458432 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d321025-4861-4cbb-8690-2bbe10e1c329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:19.524813 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-8d321025-4861-4cbb-8690-2bbe10e1c329 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:19.526217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8d321025-4861-4cbb-8690-2bbe10e1c329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:19.539856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:19.623779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:19.662950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ed7ac-b16a-47b2-9758-1385c1782265 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:19.664513 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2f8ed7ac-b16a-47b2-9758-1385c1782265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:19.887221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ed7ac-b16a-47b2-9758-1385c1782265 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:19.900035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:19.961229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5a473-0ceb-41a8-8582-fbdd7febb8c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:19.961845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5a473-0ceb-41a8-8582-fbdd7febb8c2 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:19.962158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5a473-0ceb-41a8-8582-fbdd7febb8c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:19.962434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5a473-0ceb-41a8-8582-fbdd7febb8c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:19.964066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-41e5a473-0ceb-41a8-8582-fbdd7febb8c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:20.072492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5a473-0ceb-41a8-8582-fbdd7febb8c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:20.073955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5a473-0ceb-41a8-8582-fbdd7febb8c2 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:20.092373 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 723/1446] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:22.492267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47829e43-0af2-41cd-9f1e-2967813be152 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:22.492830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47829e43-0af2-41cd-9f1e-2967813be152 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:22.492830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47829e43-0af2-41cd-9f1e-2967813be152 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:22.494196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:22.499479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87306cbe-0136-4e01-8445-e6f8a0423d0f None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:22.507354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87306cbe-0136-4e01-8445-e6f8a0423d0f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:22.507354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87306cbe-0136-4e01-8445-e6f8a0423d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:22.548813 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:22.567665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-87306cbe-0136-4e01-8445-e6f8a0423d0f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:22.568878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-87306cbe-0136-4e01-8445-e6f8a0423d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:22.668121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:22.707203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47829e43-0af2-41cd-9f1e-2967813be152 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:22.708796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-47829e43-0af2-41cd-9f1e-2967813be152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:22.813806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47829e43-0af2-41cd-9f1e-2967813be152 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:22.853409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:22.890788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f471a6d-5278-46cd-8c8a-747c889be430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:22.891404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f471a6d-5278-46cd-8c8a-747c889be430 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:22.891686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f471a6d-5278-46cd-8c8a-747c889be430 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:22.891957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f471a6d-5278-46cd-8c8a-747c889be430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:22.893563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1f471a6d-5278-46cd-8c8a-747c889be430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:23.002471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f471a6d-5278-46cd-8c8a-747c889be430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:23.004710 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f471a6d-5278-46cd-8c8a-747c889be430 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:23.023276 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 725/1449] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:25.154837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eaaf9-b682-4326-9ff6-78a00330025b None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:25.155410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eaaf9-b682-4326-9ff6-78a00330025b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:25.155410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eaaf9-b682-4326-9ff6-78a00330025b None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:25.156729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:25.162774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5f664-e74e-4df0-a4ed-de70b64e5db5 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:25.163182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5f664-e74e-4df0-a4ed-de70b64e5db5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:25.163327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5f664-e74e-4df0-a4ed-de70b64e5db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:25.228435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-68e5f664-e74e-4df0-a4ed-de70b64e5db5 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:25.229644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-68e5f664-e74e-4df0-a4ed-de70b64e5db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:25.250603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:25.326351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:25.366504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086eaaf9-b682-4326-9ff6-78a00330025b None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:25.368047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-086eaaf9-b682-4326-9ff6-78a00330025b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:25.469882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086eaaf9-b682-4326-9ff6-78a00330025b None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:25.490354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:25.545119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424d71f-806b-4ca9-ac15-cf6bf82954d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:25.545733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424d71f-806b-4ca9-ac15-cf6bf82954d2 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:25.546045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424d71f-806b-4ca9-ac15-cf6bf82954d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:25.546315 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424d71f-806b-4ca9-ac15-cf6bf82954d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:25.548086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0424d71f-806b-4ca9-ac15-cf6bf82954d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:25.654894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424d71f-806b-4ca9-ac15-cf6bf82954d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:25.656318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424d71f-806b-4ca9-ac15-cf6bf82954d2 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:25.674401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 726/1452] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:27.739760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e73b-ee52-40e6-ae68-8ca7617a642c None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:27.740294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e73b-ee52-40e6-ae68-8ca7617a642c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:27.740294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e73b-ee52-40e6-ae68-8ca7617a642c None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:27.741604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:27.746947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad85f55-d0ec-4f99-95f6-b48bcd0653ef None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:27.747355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad85f55-d0ec-4f99-95f6-b48bcd0653ef None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:27.747506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad85f55-d0ec-4f99-95f6-b48bcd0653ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:27.810919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:27.813509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1ad85f55-d0ec-4f99-95f6-b48bcd0653ef None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:27.814753 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1ad85f55-d0ec-4f99-95f6-b48bcd0653ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:27.911774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:27.950612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e73b-ee52-40e6-ae68-8ca7617a642c None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:27.952204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ab77e73b-ee52-40e6-ae68-8ca7617a642c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:28.056064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e73b-ee52-40e6-ae68-8ca7617a642c None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:28.104945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:28.130884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b0d92-f876-44d8-8a6c-7c8c4bcecb30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:28.131572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b0d92-f876-44d8-8a6c-7c8c4bcecb30 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:28.132099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b0d92-f876-44d8-8a6c-7c8c4bcecb30 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:28.132539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b0d92-f876-44d8-8a6c-7c8c4bcecb30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:28.134497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e01b0d92-f876-44d8-8a6c-7c8c4bcecb30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:28.243797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b0d92-f876-44d8-8a6c-7c8c4bcecb30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:28.245211 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b0d92-f876-44d8-8a6c-7c8c4bcecb30 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:28.263988 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 728/1455] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:30.337209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e568f9a-892b-4bd7-aab3-ab343cdae0ce None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:30.337811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e568f9a-892b-4bd7-aab3-ab343cdae0ce None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:30.337811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e568f9a-892b-4bd7-aab3-ab343cdae0ce None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:30.339120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:30.344308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f74c4a-1890-4593-81d7-7ba52c2caf33 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:30.355458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f74c4a-1890-4593-81d7-7ba52c2caf33 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:30.355458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f74c4a-1890-4593-81d7-7ba52c2caf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:30.411844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-b7f74c4a-1890-4593-81d7-7ba52c2caf33 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:30.412991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b7f74c4a-1890-4593-81d7-7ba52c2caf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:30.442876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:30.510181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:30.549416 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e568f9a-892b-4bd7-aab3-ab343cdae0ce None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:30.551713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1e568f9a-892b-4bd7-aab3-ab343cdae0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:30.655408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e568f9a-892b-4bd7-aab3-ab343cdae0ce None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:30.683616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:30.728315 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37d4c5-b64f-46b3-a0d8-48a6c9f003e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:30.728920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37d4c5-b64f-46b3-a0d8-48a6c9f003e7 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:30.729195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37d4c5-b64f-46b3-a0d8-48a6c9f003e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:30.729466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37d4c5-b64f-46b3-a0d8-48a6c9f003e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:30.731116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6f37d4c5-b64f-46b3-a0d8-48a6c9f003e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:30.836583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37d4c5-b64f-46b3-a0d8-48a6c9f003e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:30.838067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37d4c5-b64f-46b3-a0d8-48a6c9f003e7 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:30.856390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 729/1458] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:32.889695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d6f63-066a-4983-8913-3d842d4841b1 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:32.890253 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d6f63-066a-4983-8913-3d842d4841b1 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:32.890369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d6f63-066a-4983-8913-3d842d4841b1 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:32.891685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:32.897602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50445e89-6f65-4524-9cde-aa969ff773f4 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:32.898050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50445e89-6f65-4524-9cde-aa969ff773f4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:32.898182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50445e89-6f65-4524-9cde-aa969ff773f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:32.963310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-50445e89-6f65-4524-9cde-aa969ff773f4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:32.964510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-50445e89-6f65-4524-9cde-aa969ff773f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:32.973270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:33.059706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:33.098420 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d6f63-066a-4983-8913-3d842d4841b1 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:33.099991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-875d6f63-066a-4983-8913-3d842d4841b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:33.201715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d6f63-066a-4983-8913-3d842d4841b1 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:33.257473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:33.276237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939be8b-8f4c-493f-8e94-84bbbcbfb8cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:33.276849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939be8b-8f4c-493f-8e94-84bbbcbfb8cb service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:33.277131 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939be8b-8f4c-493f-8e94-84bbbcbfb8cb service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:33.277405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939be8b-8f4c-493f-8e94-84bbbcbfb8cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:33.279056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5939be8b-8f4c-493f-8e94-84bbbcbfb8cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:33.391322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939be8b-8f4c-493f-8e94-84bbbcbfb8cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:33.392749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939be8b-8f4c-493f-8e94-84bbbcbfb8cb service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:33.412204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 731/1461] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:35.475904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456bbe7-ca1b-4371-9079-2f414ef6fef9 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:35.476482 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456bbe7-ca1b-4371-9079-2f414ef6fef9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:35.476482 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456bbe7-ca1b-4371-9079-2f414ef6fef9 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:35.477778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:35.483126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d865c-852c-4262-abf5-d6e09ae121ca None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:35.483538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d865c-852c-4262-abf5-d6e09ae121ca None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:35.483686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d865c-852c-4262-abf5-d6e09ae121ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:35.553174 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-567d865c-852c-4262-abf5-d6e09ae121ca None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:35.554467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-567d865c-852c-4262-abf5-d6e09ae121ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:35.592008 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:35.650620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:35.689028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456bbe7-ca1b-4371-9079-2f414ef6fef9 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:35.691136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3456bbe7-ca1b-4371-9079-2f414ef6fef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:35.793320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456bbe7-ca1b-4371-9079-2f414ef6fef9 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:35.832368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:35.868721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04870b81-ba1c-4e71-a5bd-adbbeb13594e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:35.869340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04870b81-ba1c-4e71-a5bd-adbbeb13594e service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:35.869627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04870b81-ba1c-4e71-a5bd-adbbeb13594e service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:35.869902 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04870b81-ba1c-4e71-a5bd-adbbeb13594e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:35.871562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-04870b81-ba1c-4e71-a5bd-adbbeb13594e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:35.980035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04870b81-ba1c-4e71-a5bd-adbbeb13594e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:35.981501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04870b81-ba1c-4e71-a5bd-adbbeb13594e service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:35.999486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 732/1464] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:38.013246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dbb25-daaa-45e5-a11d-c42cb65cc1ea None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:38.013811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dbb25-daaa-45e5-a11d-c42cb65cc1ea None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:38.013811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dbb25-daaa-45e5-a11d-c42cb65cc1ea None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:38.015162 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:38.020456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421fc1-3d3f-4844-8e72-06f73c5e64b3 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:38.020868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421fc1-3d3f-4844-8e72-06f73c5e64b3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:38.021011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421fc1-3d3f-4844-8e72-06f73c5e64b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:38.085417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-88421fc1-3d3f-4844-8e72-06f73c5e64b3 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:38.086640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-88421fc1-3d3f-4844-8e72-06f73c5e64b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:38.104643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:38.184125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:38.220783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597dbb25-daaa-45e5-a11d-c42cb65cc1ea None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:38.222994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-597dbb25-daaa-45e5-a11d-c42cb65cc1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:38.325485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597dbb25-daaa-45e5-a11d-c42cb65cc1ea None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:38.392822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:38.399800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e521-bd0c-42ef-b225-8bcdab9b15a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:38.400413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e521-bd0c-42ef-b225-8bcdab9b15a8 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:38.400690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e521-bd0c-42ef-b225-8bcdab9b15a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:38.400957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e521-bd0c-42ef-b225-8bcdab9b15a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:38.402655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5701e521-bd0c-42ef-b225-8bcdab9b15a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:38.510804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701e521-bd0c-42ef-b225-8bcdab9b15a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:38.512224 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701e521-bd0c-42ef-b225-8bcdab9b15a8 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:38.532573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 734/1467] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:40.579414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aa9cf-af8c-465e-b23e-4c9ee99c2852 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:40.580017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aa9cf-af8c-465e-b23e-4c9ee99c2852 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:40.580017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aa9cf-af8c-465e-b23e-4c9ee99c2852 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:40.581766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:34:40.587579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c616671-bd8d-4fa1-a8da-bb2f15dcc56a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:40.588120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c616671-bd8d-4fa1-a8da-bb2f15dcc56a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:40.588220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c616671-bd8d-4fa1-a8da-bb2f15dcc56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:40.657732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-8c616671-bd8d-4fa1-a8da-bb2f15dcc56a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:34:40.665855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8c616671-bd8d-4fa1-a8da-bb2f15dcc56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:40.712249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:40.762637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:34:40.800026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86aa9cf-af8c-465e-b23e-4c9ee99c2852 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:34:40.801575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f86aa9cf-af8c-465e-b23e-4c9ee99c2852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:40.906811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86aa9cf-af8c-465e-b23e-4c9ee99c2852 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:34:40.953510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:40.979591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc10a32-7882-4231-bad2-63c59f7aef16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:34:40.980194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc10a32-7882-4231-bad2-63c59f7aef16 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:34:40.980476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc10a32-7882-4231-bad2-63c59f7aef16 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:34:40.980749 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc10a32-7882-4231-bad2-63c59f7aef16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:34:40.982427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9fc10a32-7882-4231-bad2-63c59f7aef16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:34:41.090109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc10a32-7882-4231-bad2-63c59f7aef16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:34:41.091540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc10a32-7882-4231-bad2-63c59f7aef16 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:34:41.109738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 735/1470] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:34:50.760522 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=22 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:34:51.107467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=66 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:00.774177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34b459-82d8-4c66-87a1-97e403b866b9 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:00.774751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34b459-82d8-4c66-87a1-97e403b866b9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:00.774751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34b459-82d8-4c66-87a1-97e403b866b9 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:00.775995 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 736/1471] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:35:00.781216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc6e47-4255-47ff-8b8c-d7ab09ba6ab6 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:00.781639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc6e47-4255-47ff-8b8c-d7ab09ba6ab6 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:00.781773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc6e47-4255-47ff-8b8c-d7ab09ba6ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:00.847114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ebfc6e47-4255-47ff-8b8c-d7ab09ba6ab6 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:35:00.848331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ebfc6e47-4255-47ff-8b8c-d7ab09ba6ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:00.942264 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 736/1472] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:35:01.026131 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ad9da-9396-4c92-b4d9-ae90b7beef97 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:01.026366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ad9da-9396-4c92-b4d9-ae90b7beef97 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:01.026635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ad9da-9396-4c92-b4d9-ae90b7beef97 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:01.027950 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 737/1473] 10.222.0.73 () {54 vars in 966 bytes} [Thu Aug 4 21:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:35:01.033044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c67e09-d928-4095-9ee5-9bef38cd208b None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:01.033360 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c67e09-d928-4095-9ee5-9bef38cd208b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:01.033637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c67e09-d928-4095-9ee5-9bef38cd208b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:01.107709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-57c67e09-d928-4095-9ee5-9bef38cd208b None None] MFA Rules not processed for user `03a9a6bcfa1b469298774e09a8912b93`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:35:01.108913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-57c67e09-d928-4095-9ee5-9bef38cd208b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:01.130014 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:01.184878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 737/1474] 10.222.0.73 () {58 vars in 1063 bytes} [Thu Aug 4 21:35:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:35:01.192926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ad9da-9396-4c92-b4d9-ae90b7beef97 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:01.194496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c75ad9da-9396-4c92-b4d9-ae90b7beef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:01.297646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ad9da-9396-4c92-b4d9-ae90b7beef97 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:01.369295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44face-8715-44f8-be32-d92623b1a2b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a9a6bcfa1b469298774e09a8912b93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:01.369903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44face-8715-44f8-be32-d92623b1a2b1 service glance] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:01.370226 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44face-8715-44f8-be32-d92623b1a2b1 service glance] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:01.370483 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44face-8715-44f8-be32-d92623b1a2b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:01.372083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1f44face-8715-44f8-be32-d92623b1a2b1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:01.425490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:01.479634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44face-8715-44f8-be32-d92623b1a2b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:01.481048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44face-8715-44f8-be32-d92623b1a2b1 service glance] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:01.499886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 738/1475] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:03.614929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1f62c-d34b-4cb9-8752-ce2cb0a4cae4 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:03.615462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1f62c-d34b-4cb9-8752-ce2cb0a4cae4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:03.615462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1f62c-d34b-4cb9-8752-ce2cb0a4cae4 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:03.616746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 738/1476] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:35:03.621881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2ac52-2ce1-4c47-8e32-a66612c928b4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:03.622357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2ac52-2ce1-4c47-8e32-a66612c928b4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:03.622469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2ac52-2ce1-4c47-8e32-a66612c928b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:03.694490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-7da2ac52-2ce1-4c47-8e32-a66612c928b4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:35:03.695699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7da2ac52-2ce1-4c47-8e32-a66612c928b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:03.795187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 739/1477] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:35:03.818523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1f62c-d34b-4cb9-8752-ce2cb0a4cae4 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:03.820066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a8c1f62c-d34b-4cb9-8752-ce2cb0a4cae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:03.945408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1f62c-d34b-4cb9-8752-ce2cb0a4cae4 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:03.993543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:04.037792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9639ad-5581-4454-82ba-f0e5b58f3d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:04.038444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9639ad-5581-4454-82ba-f0e5b58f3d16 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:04.038726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9639ad-5581-4454-82ba-f0e5b58f3d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:04.038997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9639ad-5581-4454-82ba-f0e5b58f3d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:04.039682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9639ad-5581-4454-82ba-f0e5b58f3d16 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:04.041412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: WARNING py.warnings [None req-6f9639ad-5581-4454-82ba-f0e5b58f3d16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:35:04.041412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: warnings.warn(msg) Aug 04 21:35:04.041412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: Aug 04 21:35:04.042578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9639ad-5581-4454-82ba-f0e5b58f3d16 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:04.060998 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 739/1478] 10.222.0.73 () {56 vars in 1108 bytes} [Thu Aug 4 21:35:03 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:35:06.087434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eab1f1-782c-4a1d-b9be-013fb611691e None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:06.087986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eab1f1-782c-4a1d-b9be-013fb611691e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:06.087986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eab1f1-782c-4a1d-b9be-013fb611691e None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:06.089264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 740/1479] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:35:06.095103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b03ce-1de3-4812-8422-74a69388805d None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:06.095608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b03ce-1de3-4812-8422-74a69388805d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:06.095953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b03ce-1de3-4812-8422-74a69388805d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:06.162794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-077b03ce-1de3-4812-8422-74a69388805d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:35:06.163994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-077b03ce-1de3-4812-8422-74a69388805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:06.264597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 740/1480] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:35:06.307507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eab1f1-782c-4a1d-b9be-013fb611691e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:06.309037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-99eab1f1-782c-4a1d-b9be-013fb611691e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:06.319545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:06.455271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eab1f1-782c-4a1d-b9be-013fb611691e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:06.534976 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:06.567192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a77a76-4235-43b8-b342-3331fc7e9bba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:06.567810 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a77a76-4235-43b8-b342-3331fc7e9bba service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:06.568093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a77a76-4235-43b8-b342-3331fc7e9bba service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:06.568385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a77a76-4235-43b8-b342-3331fc7e9bba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:06.570062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-34a77a76-4235-43b8-b342-3331fc7e9bba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:06.660553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a77a76-4235-43b8-b342-3331fc7e9bba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:06.662175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a77a76-4235-43b8-b342-3331fc7e9bba service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:06.679923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 741/1481] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:08.868565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952edc65-6a19-4f53-807f-0a6a5bfa016e None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:08.869093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952edc65-6a19-4f53-807f-0a6a5bfa016e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:08.869093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952edc65-6a19-4f53-807f-0a6a5bfa016e None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:08.870416 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 741/1482] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:35:08.876194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e197034-3d2a-40f6-af72-baf0866acf6b None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:08.876613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e197034-3d2a-40f6-af72-baf0866acf6b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:08.876877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e197034-3d2a-40f6-af72-baf0866acf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:08.941503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-7e197034-3d2a-40f6-af72-baf0866acf6b None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:35:08.942718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7e197034-3d2a-40f6-af72-baf0866acf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:08.989826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:09.039463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 742/1483] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:35:09.076113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952edc65-6a19-4f53-807f-0a6a5bfa016e None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:09.077658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-952edc65-6a19-4f53-807f-0a6a5bfa016e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:09.100762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:09.179410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952edc65-6a19-4f53-807f-0a6a5bfa016e None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:09.251037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2fc422-3bec-4dce-a901-bb8d9e911c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:09.251642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fc422-3bec-4dce-a901-bb8d9e911c44 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:09.251918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fc422-3bec-4dce-a901-bb8d9e911c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:09.252186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fc422-3bec-4dce-a901-bb8d9e911c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:09.253915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bf2fc422-3bec-4dce-a901-bb8d9e911c44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:09.328543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:09.365409 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2fc422-3bec-4dce-a901-bb8d9e911c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:09.366877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2fc422-3bec-4dce-a901-bb8d9e911c44 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:09.386914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 742/1484] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:11.457566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d989132-de81-4ad4-b704-8a41ffd550de None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:11.458252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d989132-de81-4ad4-b704-8a41ffd550de None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:11.458252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d989132-de81-4ad4-b704-8a41ffd550de None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:11.459589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 743/1485] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:35:11.464756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09e8f8-fd8f-45bb-8c44-645e5e1d52e4 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:11.465190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09e8f8-fd8f-45bb-8c44-645e5e1d52e4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:11.465362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09e8f8-fd8f-45bb-8c44-645e5e1d52e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:11.533601 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-4e09e8f8-fd8f-45bb-8c44-645e5e1d52e4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:35:11.534849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4e09e8f8-fd8f-45bb-8c44-645e5e1d52e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:11.630284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 743/1486] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:35:11.670508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d989132-de81-4ad4-b704-8a41ffd550de None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:11.672161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7d989132-de81-4ad4-b704-8a41ffd550de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:11.777543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d989132-de81-4ad4-b704-8a41ffd550de None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:11.847952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:11.849674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf09f1-2ae9-45fb-8c1a-20f3ce1903e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:11.850312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf09f1-2ae9-45fb-8c1a-20f3ce1903e0 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:11.850593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf09f1-2ae9-45fb-8c1a-20f3ce1903e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:11.850865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf09f1-2ae9-45fb-8c1a-20f3ce1903e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:11.852464 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-96bf09f1-2ae9-45fb-8c1a-20f3ce1903e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:11.959682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf09f1-2ae9-45fb-8c1a-20f3ce1903e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:11.961101 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf09f1-2ae9-45fb-8c1a-20f3ce1903e0 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:11.978586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 744/1487] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:14.141826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e580-aed5-4c2d-bc42-681c903dcb64 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:14.142356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e580-aed5-4c2d-bc42-681c903dcb64 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:14.142509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e580-aed5-4c2d-bc42-681c903dcb64 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:14.143857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 744/1488] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:35:14.149140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc45d1-fbe6-4b3d-a650-28d2e2539d78 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:14.149597 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc45d1-fbe6-4b3d-a650-28d2e2539d78 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:14.149707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc45d1-fbe6-4b3d-a650-28d2e2539d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:14.214953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-a0cc45d1-fbe6-4b3d-a650-28d2e2539d78 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:35:14.216153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a0cc45d1-fbe6-4b3d-a650-28d2e2539d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:14.270799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:14.314653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 745/1489] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:35:14.356063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e580-aed5-4c2d-bc42-681c903dcb64 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:14.357610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-fe02e580-aed5-4c2d-bc42-681c903dcb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:14.394366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:14.461915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e580-aed5-4c2d-bc42-681c903dcb64 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:14.538835 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb986c-501c-4553-a1c1-d4ab73eb493f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:14.539454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb986c-501c-4553-a1c1-d4ab73eb493f service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:14.539739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb986c-501c-4553-a1c1-d4ab73eb493f service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:14.540014 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb986c-501c-4553-a1c1-d4ab73eb493f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:14.541654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-27cb986c-501c-4553-a1c1-d4ab73eb493f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:14.622266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:14.654451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb986c-501c-4553-a1c1-d4ab73eb493f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:14.655810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb986c-501c-4553-a1c1-d4ab73eb493f service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:14.673558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 745/1490] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:17.045625 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a5c44-c2e7-4f7c-b9e4-29b67378feca None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:17.046225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a5c44-c2e7-4f7c-b9e4-29b67378feca None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:17.046225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a5c44-c2e7-4f7c-b9e4-29b67378feca None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:17.047757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 746/1491] 10.222.0.73 () {54 vars in 927 bytes} [Thu Aug 4 21:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:35:17.052907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f04f97-b275-4585-9c7b-c015c335cfa4 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:17.053311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f04f97-b275-4585-9c7b-c015c335cfa4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:17.053467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f04f97-b275-4585-9c7b-c015c335cfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:17.119596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-35f04f97-b275-4585-9c7b-c015c335cfa4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:35:17.120813 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-35f04f97-b275-4585-9c7b-c015c335cfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:17.217565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 746/1492] 10.222.0.73 () {58 vars in 1024 bytes} [Thu Aug 4 21:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:35:17.258346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a5c44-c2e7-4f7c-b9e4-29b67378feca None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:35:17.259909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e40a5c44-c2e7-4f7c-b9e4-29b67378feca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:17.362701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a5c44-c2e7-4f7c-b9e4-29b67378feca None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:35:17.436303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5ace9-59ea-47ee-bbdc-70db93ba7d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:35:17.436917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5ace9-59ea-47ee-bbdc-70db93ba7d43 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:35:17.437199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5ace9-59ea-47ee-bbdc-70db93ba7d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:35:17.437491 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5ace9-59ea-47ee-bbdc-70db93ba7d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:35:17.439811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4df5ace9-59ea-47ee-bbdc-70db93ba7d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:35:17.448831 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:17.547946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5ace9-59ea-47ee-bbdc-70db93ba7d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:35:17.551170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5ace9-59ea-47ee-bbdc-70db93ba7d43 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:35:17.567073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 747/1493] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:35:27.215753 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=82 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:35:27.564809 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=50 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:36.857421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2404fb-a535-40e6-9461-320da4fab176 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:36.857996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2404fb-a535-40e6-9461-320da4fab176 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:36.857996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2404fb-a535-40e6-9461-320da4fab176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:36.935510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-bf2404fb-a535-40e6-9461-320da4fab176 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:36:36.936737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bf2404fb-a535-40e6-9461-320da4fab176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:37.031301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 747/1494] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:36:37.039544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008295e1-44f8-4b52-9bb3-583c0232d3f2 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:37.040060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008295e1-44f8-4b52-9bb3-583c0232d3f2 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:37.040163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008295e1-44f8-4b52-9bb3-583c0232d3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:37.116152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-008295e1-44f8-4b52-9bb3-583c0232d3f2 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:36:37.117280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-008295e1-44f8-4b52-9bb3-583c0232d3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:37.212289 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 748/1495] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:36:37.222704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2404fb-a535-40e6-9461-320da4fab176 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:37.223967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bf2404fb-a535-40e6-9461-320da4fab176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:37.236698 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2404fb-a535-40e6-9461-320da4fab176 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:37.298140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:37.331579 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03763e5f-7841-4008-835d-1a220a127003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:37.332186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03763e5f-7841-4008-835d-1a220a127003 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:37.332462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03763e5f-7841-4008-835d-1a220a127003 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:37.332728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03763e5f-7841-4008-835d-1a220a127003 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:37.333406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03763e5f-7841-4008-835d-1a220a127003 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:36:37.333703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03763e5f-7841-4008-835d-1a220a127003 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:37.334743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: WARNING py.warnings [None req-03763e5f-7841-4008-835d-1a220a127003 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:36:37.334743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: warnings.warn(msg) Aug 04 21:36:37.334743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: Aug 04 21:36:37.335621 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03763e5f-7841-4008-835d-1a220a127003 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:37.350333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 748/1496] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:36:37.355092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008295e1-44f8-4b52-9bb3-583c0232d3f2 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:37.364603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008295e1-44f8-4b52-9bb3-583c0232d3f2 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:37.425959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:37.460318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e8f70-2f4f-4362-a765-c4d34c9e16fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:37.460937 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e8f70-2f4f-4362-a765-c4d34c9e16fc admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:37.461214 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e8f70-2f4f-4362-a765-c4d34c9e16fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:37.461483 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e8f70-2f4f-4362-a765-c4d34c9e16fc admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:37.462170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e8f70-2f4f-4362-a765-c4d34c9e16fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:37.463981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: WARNING py.warnings [None req-1f0e8f70-2f4f-4362-a765-c4d34c9e16fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:36:37.463981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: warnings.warn(msg) Aug 04 21:36:37.463981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: Aug 04 21:36:37.465163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e8f70-2f4f-4362-a765-c4d34c9e16fc admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:37.599900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 749/1497] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:36:37 2022] POST /identity/v3/projects => generated 377 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:36:37.604183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03763e5f-7841-4008-835d-1a220a127003 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:37.605760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-03763e5f-7841-4008-835d-1a220a127003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:37.641387 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03763e5f-7841-4008-835d-1a220a127003 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:37.733519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d75d69-8f9c-46a0-9987-0666bc735e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:37.735128 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d75d69-8f9c-46a0-9987-0666bc735e0b admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:37.735512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d75d69-8f9c-46a0-9987-0666bc735e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:37.735791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d75d69-8f9c-46a0-9987-0666bc735e0b admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:37.736464 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d75d69-8f9c-46a0-9987-0666bc735e0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:37.737481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: WARNING py.warnings [None req-33d75d69-8f9c-46a0-9987-0666bc735e0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:36:37.737481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: warnings.warn(msg) Aug 04 21:36:37.737481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: Aug 04 21:36:37.738727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d75d69-8f9c-46a0-9987-0666bc735e0b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:37.846555 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 749/1498] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:36:37 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:36:37.850793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e8f70-2f4f-4362-a765-c4d34c9e16fc admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:37.852321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1f0e8f70-2f4f-4362-a765-c4d34c9e16fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:37.916333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e8f70-2f4f-4362-a765-c4d34c9e16fc admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:38.000453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:38.010856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fda52-f2ab-4641-b8fb-ca1927b9e59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:38.011543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fda52-f2ab-4641-b8fb-ca1927b9e59b admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:38.011862 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fda52-f2ab-4641-b8fb-ca1927b9e59b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:38.012141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fda52-f2ab-4641-b8fb-ca1927b9e59b admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:38.012891 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fda52-f2ab-4641-b8fb-ca1927b9e59b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:38.013906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: WARNING py.warnings [None req-430fda52-f2ab-4641-b8fb-ca1927b9e59b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:36:38.013906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: warnings.warn(msg) Aug 04 21:36:38.013906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: Aug 04 21:36:38.014836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fda52-f2ab-4641-b8fb-ca1927b9e59b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:38.027451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 750/1499] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:36:38.032097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d75d69-8f9c-46a0-9987-0666bc735e0b admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:38.033703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-33d75d69-8f9c-46a0-9987-0666bc735e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:38.045377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d75d69-8f9c-46a0-9987-0666bc735e0b admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:38.071602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:38.141936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b4a07-f7da-434c-b851-8228856de562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:38.142590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4a07-f7da-434c-b851-8228856de562 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:38.142878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4a07-f7da-434c-b851-8228856de562 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:38.143147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4a07-f7da-434c-b851-8228856de562 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:38.143841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b4a07-f7da-434c-b851-8228856de562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:38.144860 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: WARNING py.warnings [None req-575b4a07-f7da-434c-b851-8228856de562 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:36:38.144860 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: warnings.warn(msg) Aug 04 21:36:38.144860 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: Aug 04 21:36:38.145745 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b4a07-f7da-434c-b851-8228856de562 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:38.158866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 750/1500] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:36:38.163014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fda52-f2ab-4641-b8fb-ca1927b9e59b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:38.164530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-430fda52-f2ab-4641-b8fb-ca1927b9e59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:38.198331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fda52-f2ab-4641-b8fb-ca1927b9e59b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:38.293977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ad1cb-ae2e-4254-89ae-d23d84bc5941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:38.294596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ad1cb-ae2e-4254-89ae-d23d84bc5941 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:38.294881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ad1cb-ae2e-4254-89ae-d23d84bc5941 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:38.295159 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ad1cb-ae2e-4254-89ae-d23d84bc5941 admin admin] PATH_INFO: `/v3/projects/7767b6b5264248d8a5f8d9564dac6274/users/52edb76d922049c08a8980f040606a85/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:38.317919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ad1cb-ae2e-4254-89ae-d23d84bc5941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7767b6b5264248d8a5f8d9564dac6274, user_id=52edb76d922049c08a8980f040606a85, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:38.318967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: WARNING py.warnings [None req-da1ad1cb-ae2e-4254-89ae-d23d84bc5941 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:36:38.318967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: warnings.warn(msg) Aug 04 21:36:38.318967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: Aug 04 21:36:38.320901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ad1cb-ae2e-4254-89ae-d23d84bc5941 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:38.408587 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 751/1501] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:36:38 2022] PUT /identity/v3/projects/7767b6b5264248d8a5f8d9564dac6274/users/52edb76d922049c08a8980f040606a85/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:36:38.420729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa7818-a7ab-4596-a3a2-c9c0cbd42070 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:38.421017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa7818-a7ab-4596-a3a2-c9c0cbd42070 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:38.421294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa7818-a7ab-4596-a3a2-c9c0cbd42070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:38.478673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-acfa7818-a7ab-4596-a3a2-c9c0cbd42070 None None] MFA Rules not processed for user `52edb76d922049c08a8980f040606a85`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:36:38.479895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-acfa7818-a7ab-4596-a3a2-c9c0cbd42070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:38.560659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 751/1502] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:36:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:36:38.573717 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953792f5-61c4-4cab-ab34-d53b3d813977 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:38.574019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953792f5-61c4-4cab-ab34-d53b3d813977 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:38.574314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953792f5-61c4-4cab-ab34-d53b3d813977 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:38.575651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 752/1503] 10.222.0.73 () {54 vars in 964 bytes} [Thu Aug 4 21:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:36:38.581298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec43268-de1b-4f81-a2e2-47b2d1139e36 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:38.581592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec43268-de1b-4f81-a2e2-47b2d1139e36 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:38.581874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec43268-de1b-4f81-a2e2-47b2d1139e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:38.613439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:38.659902 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-8ec43268-de1b-4f81-a2e2-47b2d1139e36 None None] MFA Rules not processed for user `f3264e17413e4c64b644c1b607cca15b`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:36:38.661104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8ec43268-de1b-4f81-a2e2-47b2d1139e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:38.737893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 752/1504] 10.222.0.73 () {58 vars in 1061 bytes} [Thu Aug 4 21:36:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:36:38.745588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953792f5-61c4-4cab-ab34-d53b3d813977 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:38.747155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-953792f5-61c4-4cab-ab34-d53b3d813977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:38.798467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:38.852057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953792f5-61c4-4cab-ab34-d53b3d813977 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:38.926403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5aa97-6f29-49cd-a2f3-fdf50fa09012 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:38.927017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5aa97-6f29-49cd-a2f3-fdf50fa09012 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:38.927299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5aa97-6f29-49cd-a2f3-fdf50fa09012 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:38.927572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5aa97-6f29-49cd-a2f3-fdf50fa09012 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:38.929190 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c9c5aa97-6f29-49cd-a2f3-fdf50fa09012 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:39.027031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5aa97-6f29-49cd-a2f3-fdf50fa09012 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:39.028502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5aa97-6f29-49cd-a2f3-fdf50fa09012 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:39.032552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:39.048054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 753/1505] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:36:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:36:39.093186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f3b03-950d-4e7d-843f-16eacf27cd76 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:39.094244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f3b03-950d-4e7d-843f-16eacf27cd76 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:39.094448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f3b03-950d-4e7d-843f-16eacf27cd76 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:39.095864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 753/1506] 10.222.0.73 () {52 vars in 830 bytes} [Thu Aug 4 21:36:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:36:39.099384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5aa97-6f29-49cd-a2f3-fdf50fa09012 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:39.100619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c9c5aa97-6f29-49cd-a2f3-fdf50fa09012 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:39.198212 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5aa97-6f29-49cd-a2f3-fdf50fa09012 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:39.291448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:39.292117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:39.292401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:39.292678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] PATH_INFO: `/v3/users/52edb76d922049c08a8980f040606a85` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:39.304714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52edb76d922049c08a8980f040606a85)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:39.305726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: WARNING py.warnings [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:36:39.305726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: warnings.warn(msg) Aug 04 21:36:39.305726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: Aug 04 21:36:39.306978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:39.316600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:39.542069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52edb76d922049c08a8980f040606a85'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:39.617668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52edb76d922049c08a8980f040606a85'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:39.642663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52edb76d922049c08a8980f040606a85'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:39.643264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52edb76d922049c08a8980f040606a85'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:39.658062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52edb76d922049c08a8980f040606a85'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:39.664814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 754/1507] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:36:39 2022] DELETE /identity/v3/users/52edb76d922049c08a8980f040606a85 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:36:39.669777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954ddf8-8100-420f-84d1-b38a4d7b7b3d None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:39.672157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954ddf8-8100-420f-84d1-b38a4d7b7b3d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:39.672157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954ddf8-8100-420f-84d1-b38a4d7b7b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:39.729804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-7954ddf8-8100-420f-84d1-b38a4d7b7b3d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:36:39.730971 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-7954ddf8-8100-420f-84d1-b38a4d7b7b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:39.806220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:39.819395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 754/1508] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:36:39.830734 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:39.832295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:39.938149 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cafa8-8c59-42dd-9aa9-95a55b410377 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:39.997558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:40.019026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c93b9a-2b8a-4e0c-a0cf-3cd61dab4a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:40.019692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c93b9a-2b8a-4e0c-a0cf-3cd61dab4a14 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:40.019980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c93b9a-2b8a-4e0c-a0cf-3cd61dab4a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:40.020256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c93b9a-2b8a-4e0c-a0cf-3cd61dab4a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:40.022068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e8c93b9a-2b8a-4e0c-a0cf-3cd61dab4a14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:40.156121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c93b9a-2b8a-4e0c-a0cf-3cd61dab4a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:40.158081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c93b9a-2b8a-4e0c-a0cf-3cd61dab4a14 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:40.178282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 755/1509] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:36:40.779962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954ddf8-8100-420f-84d1-b38a4d7b7b3d None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:36:40.781234 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-7954ddf8-8100-420f-84d1-b38a4d7b7b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:36:40.793339 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954ddf8-8100-420f-84d1-b38a4d7b7b3d None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:36:40.890554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:36:40.891607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:36:40.891890 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:36:40.892168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] PATH_INFO: `/v3/projects/7767b6b5264248d8a5f8d9564dac6274` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:36:40.902473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7767b6b5264248d8a5f8d9564dac6274)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:36:40.903513 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: WARNING py.warnings [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:36:40.903513 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: warnings.warn(msg) Aug 04 21:36:40.903513 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: Aug 04 21:36:40.904782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:36:41.013765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7767b6b5264248d8a5f8d9564dac6274', 'request_id': 'req-3934bf77-6f43-456b-8ff8-04cead7dc002'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:41.027678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7767b6b5264248d8a5f8d9564dac6274', 'request_id': 'req-3934bf77-6f43-456b-8ff8-04cead7dc002'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:41.106116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7767b6b5264248d8a5f8d9564dac6274', 'request_id': 'req-3934bf77-6f43-456b-8ff8-04cead7dc002'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:41.131977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] The token cache is being invalidate because project 7767b6b5264248d8a5f8d9564dac6274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:36:41.132268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:41.132802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3934bf77-6f43-456b-8ff8-04cead7dc002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:36:41.134210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 755/1510] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:36:40 2022] DELETE /identity/v3/projects/7767b6b5264248d8a5f8d9564dac6274 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:36:50.175682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=62 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:36:51.129837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:38:16.806609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d50ae-ce51-48ef-a6d1-bd5b523890d7 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:16.807968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d50ae-ce51-48ef-a6d1-bd5b523890d7 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:16.807968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d50ae-ce51-48ef-a6d1-bd5b523890d7 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:16.808969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:38:16.814050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8b167-b0bd-4287-a3c3-75ad9ab57b48 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:16.814487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8b167-b0bd-4287-a3c3-75ad9ab57b48 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:16.814614 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8b167-b0bd-4287-a3c3-75ad9ab57b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:16.901678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-5ec8b167-b0bd-4287-a3c3-75ad9ab57b48 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:38:16.903075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5ec8b167-b0bd-4287-a3c3-75ad9ab57b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:17.008279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:38:17.165384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d50ae-ce51-48ef-a6d1-bd5b523890d7 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:38:17.168275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-776d50ae-ce51-48ef-a6d1-bd5b523890d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:17.292924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d50ae-ce51-48ef-a6d1-bd5b523890d7 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:38:17.298571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e62ff-5989-449e-bca4-c961959b8cd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:38:17.299247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e62ff-5989-449e-bca4-c961959b8cd0 service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:17.299528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e62ff-5989-449e-bca4-c961959b8cd0 service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:17.299804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e62ff-5989-449e-bca4-c961959b8cd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:17.309257 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e62ff-5989-449e-bca4-c961959b8cd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:38:17.314360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e62ff-5989-449e-bca4-c961959b8cd0 service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:38:17.320264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 757/1513] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:38:18.152440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a969e-d070-4085-b3a4-d10ac4828960 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:18.153017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a969e-d070-4085-b3a4-d10ac4828960 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:18.153017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a969e-d070-4085-b3a4-d10ac4828960 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:18.154371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:38:18.160456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e770bd-9399-41ec-90c1-cb001a350c68 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:18.160456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e770bd-9399-41ec-90c1-cb001a350c68 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:18.160456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e770bd-9399-41ec-90c1-cb001a350c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:18.179501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-81e770bd-9399-41ec-90c1-cb001a350c68 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:38:18.180724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-81e770bd-9399-41ec-90c1-cb001a350c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:18.188822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:38:18.341895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a969e-d070-4085-b3a4-d10ac4828960 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:38:18.356322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a969e-d070-4085-b3a4-d10ac4828960 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:38:18.361645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163132e2-bfc6-467f-8763-80bad6631e82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:38:18.362346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163132e2-bfc6-467f-8763-80bad6631e82 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:18.362693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163132e2-bfc6-467f-8763-80bad6631e82 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:18.363222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163132e2-bfc6-467f-8763-80bad6631e82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:18.371889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163132e2-bfc6-467f-8763-80bad6631e82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:38:18.373470 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163132e2-bfc6-467f-8763-80bad6631e82 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:38:18.383071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 758/1516] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:38:19.808514 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885e45-9d6a-4334-a115-eb0dc4aa1d04 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:19.809080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885e45-9d6a-4334-a115-eb0dc4aa1d04 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:19.809188 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885e45-9d6a-4334-a115-eb0dc4aa1d04 None None] PATH_INFO: `/v3` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:19.810535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 759/1517] 10.222.0.95 () {54 vars in 925 bytes} [Thu Aug 4 21:38:19 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:38:19.819571 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae52a5-6382-48c7-a84c-b1f61ae7037a None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:19.820029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae52a5-6382-48c7-a84c-b1f61ae7037a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:19.820125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae52a5-6382-48c7-a84c-b1f61ae7037a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:19.879339 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-6dae52a5-6382-48c7-a84c-b1f61ae7037a None None] MFA Rules not processed for user `fab36b186fa44ee18bf72228fc7ba758`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:38:19.880526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6dae52a5-6382-48c7-a84c-b1f61ae7037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:19.897370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:38:19.951725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 759/1518] 10.222.0.95 () {58 vars in 1015 bytes} [Thu Aug 4 21:38:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:38:28.174846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=56 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:38:29.641847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f7c4a-21ad-4848-94d1-3a6739529ea6 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:29.642521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f7c4a-21ad-4848-94d1-3a6739529ea6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:29.642521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f7c4a-21ad-4848-94d1-3a6739529ea6 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:29.643764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 760/1519] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:38:29.648634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7af296-43e3-4df0-bd07-7a1753d31322 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:29.649038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7af296-43e3-4df0-bd07-7a1753d31322 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:29.649205 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7af296-43e3-4df0-bd07-7a1753d31322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:29.668258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-db7af296-43e3-4df0-bd07-7a1753d31322 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:38:29.669490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-db7af296-43e3-4df0-bd07-7a1753d31322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:29.677291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 760/1520] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:38:29.832839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f7c4a-21ad-4848-94d1-3a6739529ea6 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:38:29.847054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f7c4a-21ad-4848-94d1-3a6739529ea6 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:38:29.852671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ce4cf-33b0-4950-b2df-8c5562861db9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:38:29.853274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ce4cf-33b0-4950-b2df-8c5562861db9 service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:29.853556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ce4cf-33b0-4950-b2df-8c5562861db9 service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:29.853826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ce4cf-33b0-4950-b2df-8c5562861db9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:29.862591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ce4cf-33b0-4950-b2df-8c5562861db9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:38:29.864400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ce4cf-33b0-4950-b2df-8c5562861db9 service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:38:29.874296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 761/1521] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:38:30.690728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbf6e8-7675-4aeb-9163-1274b12bb7b5 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:30.691347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbf6e8-7675-4aeb-9163-1274b12bb7b5 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:30.691347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbf6e8-7675-4aeb-9163-1274b12bb7b5 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:30.692608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 21:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:38:30.697480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77a32e-a1ca-4a1e-b1a0-d41b59cad20a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:30.697870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77a32e-a1ca-4a1e-b1a0-d41b59cad20a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:30.701767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77a32e-a1ca-4a1e-b1a0-d41b59cad20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:30.718162 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-af77a32e-a1ca-4a1e-b1a0-d41b59cad20a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:38:30.719347 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-af77a32e-a1ca-4a1e-b1a0-d41b59cad20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:30.727200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 21:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:38:30.880675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbf6e8-7675-4aeb-9163-1274b12bb7b5 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:38:30.895971 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbf6e8-7675-4aeb-9163-1274b12bb7b5 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:38:30.901418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf5584-753b-4403-a6a9-54753e3769ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:38:30.902044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf5584-753b-4403-a6a9-54753e3769ee service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:30.902332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf5584-753b-4403-a6a9-54753e3769ee service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:30.902601 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf5584-753b-4403-a6a9-54753e3769ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:30.911502 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf5584-753b-4403-a6a9-54753e3769ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:38:30.913082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf5584-753b-4403-a6a9-54753e3769ee service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:38:30.922684 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 762/1524] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:38:30.998537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d61ae-ce31-4178-8329-7b5855de82a5 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:30.998825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d61ae-ce31-4178-8329-7b5855de82a5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:30.999099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d61ae-ce31-4178-8329-7b5855de82a5 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:31.000468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 763/1525] 10.222.0.73 () {54 vars in 961 bytes} [Thu Aug 4 21:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:38:31.005806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b656d1b-3cc0-477e-b533-fd0018b749ce None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:31.006139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b656d1b-3cc0-477e-b533-fd0018b749ce None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:31.006418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b656d1b-3cc0-477e-b533-fd0018b749ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:31.027758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-7b656d1b-3cc0-477e-b533-fd0018b749ce None None] MFA Rules not processed for user `fab36b186fa44ee18bf72228fc7ba758`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:38:31.029616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-7b656d1b-3cc0-477e-b533-fd0018b749ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:31.038334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 763/1526] 10.222.0.73 () {58 vars in 1058 bytes} [Thu Aug 4 21:38:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:38:31.046698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d61ae-ce31-4178-8329-7b5855de82a5 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:38:31.061455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d61ae-ce31-4178-8329-7b5855de82a5 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:38:31.066742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e39719-6b99-4708-bda8-8eca21df7e6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:38:31.067353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e39719-6b99-4708-bda8-8eca21df7e6d service placement] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:31.067639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e39719-6b99-4708-bda8-8eca21df7e6d service placement] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:31.067914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e39719-6b99-4708-bda8-8eca21df7e6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:31.076593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e39719-6b99-4708-bda8-8eca21df7e6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:38:31.078230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e39719-6b99-4708-bda8-8eca21df7e6d service placement] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:38:31.087585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 764/1527] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:38:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:38:34.853225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f1dd8-b77a-4abe-8a73-2a59dc28ff1e None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:34.853743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f1dd8-b77a-4abe-8a73-2a59dc28ff1e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:34.853743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f1dd8-b77a-4abe-8a73-2a59dc28ff1e None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:34.855080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:38:34.860546 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad980a6-8598-477c-a1a1-f1b8a458e2cd None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:34.861050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad980a6-8598-477c-a1a1-f1b8a458e2cd None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:34.861174 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad980a6-8598-477c-a1a1-f1b8a458e2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:34.881691 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-7ad980a6-8598-477c-a1a1-f1b8a458e2cd None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:38:34.882937 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7ad980a6-8598-477c-a1a1-f1b8a458e2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:34.891348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:38:34.963932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f1dd8-b77a-4abe-8a73-2a59dc28ff1e None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:38:34.965642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-135f1dd8-b77a-4abe-8a73-2a59dc28ff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:35.045995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f1dd8-b77a-4abe-8a73-2a59dc28ff1e None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:38:35.049669 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1da76-8468-453c-81f8-fe04db244cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:38:35.050361 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1da76-8468-453c-81f8-fe04db244cda service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:35.050644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1da76-8468-453c-81f8-fe04db244cda service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:35.050917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1da76-8468-453c-81f8-fe04db244cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:35.060086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1da76-8468-453c-81f8-fe04db244cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:38:35.061586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1da76-8468-453c-81f8-fe04db244cda service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:38:35.071093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 765/1530] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:38:37.068783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60747a0-040d-490d-a5c8-4aa9c29a53c1 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:37.069308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60747a0-040d-490d-a5c8-4aa9c29a53c1 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:37.069308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60747a0-040d-490d-a5c8-4aa9c29a53c1 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:37.070665 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:38:37.075751 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0b64-07e9-4598-bee8-50c7a77bc2c0 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:37.076194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0b64-07e9-4598-bee8-50c7a77bc2c0 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:37.076448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0b64-07e9-4598-bee8-50c7a77bc2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:37.096719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-f39c0b64-07e9-4598-bee8-50c7a77bc2c0 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:38:37.097955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f39c0b64-07e9-4598-bee8-50c7a77bc2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:37.105841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:38:37.143052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60747a0-040d-490d-a5c8-4aa9c29a53c1 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:38:37.154353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60747a0-040d-490d-a5c8-4aa9c29a53c1 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:38:37.157988 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a87d84-b127-4b81-9bcf-bb2a2922b13e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:38:37.158617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a87d84-b127-4b81-9bcf-bb2a2922b13e service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:37.158899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a87d84-b127-4b81-9bcf-bb2a2922b13e service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:37.159244 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a87d84-b127-4b81-9bcf-bb2a2922b13e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:37.168057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a87d84-b127-4b81-9bcf-bb2a2922b13e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:38:37.169520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a87d84-b127-4b81-9bcf-bb2a2922b13e service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:38:37.179529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 767/1533] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:38:40.799779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe91bc-17d7-4f12-9a44-e8eee91e47b7 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:40.800290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe91bc-17d7-4f12-9a44-e8eee91e47b7 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:40.800290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe91bc-17d7-4f12-9a44-e8eee91e47b7 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:40.801557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:38:40.806649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744a4fe-0fd6-45e6-88dc-9e5a0edb1e21 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:40.807092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744a4fe-0fd6-45e6-88dc-9e5a0edb1e21 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:40.807210 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744a4fe-0fd6-45e6-88dc-9e5a0edb1e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:40.826826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-e744a4fe-0fd6-45e6-88dc-9e5a0edb1e21 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:38:40.828025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e744a4fe-0fd6-45e6-88dc-9e5a0edb1e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:40.835846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:38:41.142400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe91bc-17d7-4f12-9a44-e8eee91e47b7 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:38:41.144103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bdbe91bc-17d7-4f12-9a44-e8eee91e47b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:41.209400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:38:41.223236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe91bc-17d7-4f12-9a44-e8eee91e47b7 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:38:41.226811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03845c5-76e8-4f78-9b62-52fe22aef7a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:38:41.227424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03845c5-76e8-4f78-9b62-52fe22aef7a5 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:41.227709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03845c5-76e8-4f78-9b62-52fe22aef7a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:41.227983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03845c5-76e8-4f78-9b62-52fe22aef7a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:41.237983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03845c5-76e8-4f78-9b62-52fe22aef7a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:38:41.239712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03845c5-76e8-4f78-9b62-52fe22aef7a5 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:38:41.249598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 768/1536] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:38:43.293014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7641d24-0ffa-4fc5-894e-be0f1d95e0b6 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:43.293588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7641d24-0ffa-4fc5-894e-be0f1d95e0b6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:43.293588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7641d24-0ffa-4fc5-894e-be0f1d95e0b6 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:43.295490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 21:38:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:38:43.301027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d05e5-c8cd-409f-a918-8401a2493e54 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:43.301493 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d05e5-c8cd-409f-a918-8401a2493e54 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:43.301622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d05e5-c8cd-409f-a918-8401a2493e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:43.322137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c82d05e5-c8cd-409f-a918-8401a2493e54 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:38:43.323354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c82d05e5-c8cd-409f-a918-8401a2493e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:43.331278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 21:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:38:43.642903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7641d24-0ffa-4fc5-894e-be0f1d95e0b6 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:38:43.644594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d7641d24-0ffa-4fc5-894e-be0f1d95e0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:38:43.658141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7641d24-0ffa-4fc5-894e-be0f1d95e0b6 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:38:43.661741 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec6492-e6e6-4e6a-a66f-af485d4045ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:38:43.662382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec6492-e6e6-4e6a-a66f-af485d4045ee service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:38:43.662659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec6492-e6e6-4e6a-a66f-af485d4045ee service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:38:43.662972 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec6492-e6e6-4e6a-a66f-af485d4045ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:38:43.673106 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec6492-e6e6-4e6a-a66f-af485d4045ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:38:43.674555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec6492-e6e6-4e6a-a66f-af485d4045ee service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:38:43.684464 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 770/1539] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:38:53.316907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=11 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:38:53.680130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=33 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:10.213029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865ad8d-7bdf-459b-bd9e-b0957f138ad3 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.214304 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865ad8d-7bdf-459b-bd9e-b0957f138ad3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.214877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865ad8d-7bdf-459b-bd9e-b0957f138ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.244979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-6865ad8d-7bdf-459b-bd9e-b0957f138ad3 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.247390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6865ad8d-7bdf-459b-bd9e-b0957f138ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.258354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 770/1540] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:10.271931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1241a9-28cd-418e-aeaf-de7878e3cc8a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.273012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1241a9-28cd-418e-aeaf-de7878e3cc8a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.273518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1241a9-28cd-418e-aeaf-de7878e3cc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.315350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-4a1241a9-28cd-418e-aeaf-de7878e3cc8a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.316716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4a1241a9-28cd-418e-aeaf-de7878e3cc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.331735 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 771/1541] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.336140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865ad8d-7bdf-459b-bd9e-b0957f138ad3 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:10.353915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865ad8d-7bdf-459b-bd9e-b0957f138ad3 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:10.363355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b35c76-3ed7-490d-8fbe-08bd26070b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:10.364347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b35c76-3ed7-490d-8fbe-08bd26070b95 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.364866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b35c76-3ed7-490d-8fbe-08bd26070b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.365437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b35c76-3ed7-490d-8fbe-08bd26070b95 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.366660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b35c76-3ed7-490d-8fbe-08bd26070b95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:10.367202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b35c76-3ed7-490d-8fbe-08bd26070b95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:10.369076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b35c76-3ed7-490d-8fbe-08bd26070b95 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:10.387577 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 771/1542] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:10.396778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1241a9-28cd-418e-aeaf-de7878e3cc8a None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:10.407518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1241a9-28cd-418e-aeaf-de7878e3cc8a None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:10.415540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951192c-ebd5-4216-84db-af1244bdf1ca None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.415869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951192c-ebd5-4216-84db-af1244bdf1ca None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.416428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951192c-ebd5-4216-84db-af1244bdf1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.421642 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e12099-e076-4186-a34d-c5c220672adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:10.422855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e12099-e076-4186-a34d-c5c220672adb admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.423136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e12099-e076-4186-a34d-c5c220672adb admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.423412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e12099-e076-4186-a34d-c5c220672adb admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.424090 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e12099-e076-4186-a34d-c5c220672adb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:10.425959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e12099-e076-4186-a34d-c5c220672adb admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:10.459474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d951192c-ebd5-4216-84db-af1244bdf1ca None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.460726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d951192c-ebd5-4216-84db-af1244bdf1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.482639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 772/1543] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.489870 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073c5fa-27c5-4bfc-9caa-b61650d10169 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.490185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073c5fa-27c5-4bfc-9caa-b61650d10169 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.490596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073c5fa-27c5-4bfc-9caa-b61650d10169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.534340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-b073c5fa-27c5-4bfc-9caa-b61650d10169 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.534340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b073c5fa-27c5-4bfc-9caa-b61650d10169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.543538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 773/1544] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.545460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac685a0-0d00-4af8-9ee9-64035603ce54 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.545958 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac685a0-0d00-4af8-9ee9-64035603ce54 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.546470 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac685a0-0d00-4af8-9ee9-64035603ce54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.568536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-cac685a0-0d00-4af8-9ee9-64035603ce54 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.570200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-cac685a0-0d00-4af8-9ee9-64035603ce54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.580961 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 774/1545] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.582453 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5a561-a551-4f13-be65-aecce8f6539f None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.582738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5a561-a551-4f13-be65-aecce8f6539f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.583012 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5a561-a551-4f13-be65-aecce8f6539f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.585032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 772/1546] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/projects => generated 405 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:10.586754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4bbf7-928b-4938-9642-04bcab3124ef None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.587339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4bbf7-928b-4938-9642-04bcab3124ef None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.588115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4bbf7-928b-4938-9642-04bcab3124ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.609621 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-99e4bbf7-928b-4938-9642-04bcab3124ef None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.610949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d8a5a561-a551-4f13-be65-aecce8f6539f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.611087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-99e4bbf7-928b-4938-9642-04bcab3124ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.612875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d8a5a561-a551-4f13-be65-aecce8f6539f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.673583 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 773/1547] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.681441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4bbf7-928b-4938-9642-04bcab3124ef None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:10.681637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 775/1548] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.681637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2367faf-f116-439f-ad5e-6fd025c082c0 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.681637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2367faf-f116-439f-ad5e-6fd025c082c0 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.681637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2367faf-f116-439f-ad5e-6fd025c082c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.692053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4bbf7-928b-4938-9642-04bcab3124ef None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:10.696526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c2367faf-f116-439f-ad5e-6fd025c082c0 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.696721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46971747-f3ae-4cd6-8f12-34b3b9d9b162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:10.697339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46971747-f3ae-4cd6-8f12-34b3b9d9b162 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.697727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c2367faf-f116-439f-ad5e-6fd025c082c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.697844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46971747-f3ae-4cd6-8f12-34b3b9d9b162 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.698022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46971747-f3ae-4cd6-8f12-34b3b9d9b162 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.698760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46971747-f3ae-4cd6-8f12-34b3b9d9b162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:10.699068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46971747-f3ae-4cd6-8f12-34b3b9d9b162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:10.700073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: WARNING py.warnings [None req-46971747-f3ae-4cd6-8f12-34b3b9d9b162 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:40:10.700073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: warnings.warn(msg) Aug 04 21:40:10.700073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: Aug 04 21:40:10.700957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46971747-f3ae-4cd6-8f12-34b3b9d9b162 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:10.706481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 776/1549] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.708079 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd33c0-8486-40f4-9b23-bd5454828c4a None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.708079 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd33c0-8486-40f4-9b23-bd5454828c4a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.708315 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd33c0-8486-40f4-9b23-bd5454828c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.714584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 774/1550] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:10.715724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fbf17-c3c8-46a3-9fd6-5105ddd1c1ac None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.716138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fbf17-c3c8-46a3-9fd6-5105ddd1c1ac None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.716321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fbf17-c3c8-46a3-9fd6-5105ddd1c1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.728126 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-f1fd33c0-8486-40f4-9b23-bd5454828c4a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.729357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f1fd33c0-8486-40f4-9b23-bd5454828c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.736043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-7c9fbf17-c3c8-46a3-9fd6-5105ddd1c1ac None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.737207 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7c9fbf17-c3c8-46a3-9fd6-5105ddd1c1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.738336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 777/1551] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.739147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd33c0-8486-40f4-9b23-bd5454828c4a None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:10.746041 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 775/1552] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.747884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cec2b-6ea8-49ab-84b2-c110ca9b7dea None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.748175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cec2b-6ea8-49ab-84b2-c110ca9b7dea None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.749155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cec2b-6ea8-49ab-84b2-c110ca9b7dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.753134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd33c0-8486-40f4-9b23-bd5454828c4a None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:10.757882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30423cda-2230-4390-87ca-c1916f4cbfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:10.758541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30423cda-2230-4390-87ca-c1916f4cbfcd admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.758826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30423cda-2230-4390-87ca-c1916f4cbfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.759105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30423cda-2230-4390-87ca-c1916f4cbfcd admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.759800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30423cda-2230-4390-87ca-c1916f4cbfcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:10.762496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30423cda-2230-4390-87ca-c1916f4cbfcd admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:10.768794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-038cec2b-6ea8-49ab-84b2-c110ca9b7dea None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.769978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-038cec2b-6ea8-49ab-84b2-c110ca9b7dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.778451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 776/1553] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.779636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9b304-8160-4e66-98ed-6a908287ab18 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.779923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9b304-8160-4e66-98ed-6a908287ab18 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.780198 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9b304-8160-4e66-98ed-6a908287ab18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.799598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-a0c9b304-8160-4e66-98ed-6a908287ab18 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.800779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a0c9b304-8160-4e66-98ed-6a908287ab18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.809255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 777/1554] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.811344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31560d6-d745-45ca-b908-324a034ce1b6 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.811344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31560d6-d745-45ca-b908-324a034ce1b6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.811344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31560d6-d745-45ca-b908-324a034ce1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.829304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c31560d6-d745-45ca-b908-324a034ce1b6 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.830516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c31560d6-d745-45ca-b908-324a034ce1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.838311 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 778/1555] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.842110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257e693-465a-457d-ad53-32e492e94525 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.842110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257e693-465a-457d-ad53-32e492e94525 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.842110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257e693-465a-457d-ad53-32e492e94525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.859266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-2257e693-465a-457d-ad53-32e492e94525 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.860432 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2257e693-465a-457d-ad53-32e492e94525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.868569 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 779/1556] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.869302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257e693-465a-457d-ad53-32e492e94525 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:10.874205 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 778/1557] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/users => generated 303 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:10.875666 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd95c0-d8f6-4a4f-9a16-16b7ce91fa80 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.875953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd95c0-d8f6-4a4f-9a16-16b7ce91fa80 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.876229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd95c0-d8f6-4a4f-9a16-16b7ce91fa80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.880200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257e693-465a-457d-ad53-32e492e94525 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:10.888215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955c674-13e6-4177-af03-682bf880f78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:10.888215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c674-13e6-4177-af03-682bf880f78e admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.888215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c674-13e6-4177-af03-682bf880f78e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.888215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c674-13e6-4177-af03-682bf880f78e admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.888628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955c674-13e6-4177-af03-682bf880f78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:10.888628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955c674-13e6-4177-af03-682bf880f78e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:10.895429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-48dd95c0-d8f6-4a4f-9a16-16b7ce91fa80 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.896625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-48dd95c0-d8f6-4a4f-9a16-16b7ce91fa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.904847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 779/1558] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.905643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd95c0-d8f6-4a4f-9a16-16b7ce91fa80 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:10.915001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd95c0-d8f6-4a4f-9a16-16b7ce91fa80 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:10.929186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689e6ee-fd4f-4489-bd74-bdc4b3120976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:10.929186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689e6ee-fd4f-4489-bd74-bdc4b3120976 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.929186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689e6ee-fd4f-4489-bd74-bdc4b3120976 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.929186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689e6ee-fd4f-4489-bd74-bdc4b3120976 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.929613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689e6ee-fd4f-4489-bd74-bdc4b3120976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:10.929613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689e6ee-fd4f-4489-bd74-bdc4b3120976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:10.929613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689e6ee-fd4f-4489-bd74-bdc4b3120976 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:10.935903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 780/1559] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:10.937052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b056f44-2747-4698-b8a0-5b80c62840b3 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.937341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b056f44-2747-4698-b8a0-5b80c62840b3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.937620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b056f44-2747-4698-b8a0-5b80c62840b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.957806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0b056f44-2747-4698-b8a0-5b80c62840b3 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:10.959046 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0b056f44-2747-4698-b8a0-5b80c62840b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:10.967072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 781/1560] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:10.967840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b056f44-2747-4698-b8a0-5b80c62840b3 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:10.976764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b056f44-2747-4698-b8a0-5b80c62840b3 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:10.981957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a8db0-ab6f-4306-b04b-18c75fd6449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:10.986607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a8db0-ab6f-4306-b04b-18c75fd6449a admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:10.986607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a8db0-ab6f-4306-b04b-18c75fd6449a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:10.986607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a8db0-ab6f-4306-b04b-18c75fd6449a admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:10.986607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a8db0-ab6f-4306-b04b-18c75fd6449a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:10.986607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a8db0-ab6f-4306-b04b-18c75fd6449a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:10.986607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a8db0-ab6f-4306-b04b-18c75fd6449a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:10.998070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 782/1561] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:10.998841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a8db0-ab6f-4306-b04b-18c75fd6449a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.010176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a8db0-ab6f-4306-b04b-18c75fd6449a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.013471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 780/1562] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:10 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:11.013471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955c674-13e6-4177-af03-682bf880f78e admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.062249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:11.072180 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab52d8-ae11-4c1f-9db1-99bdcf96e29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.072820 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab52d8-ae11-4c1f-9db1-99bdcf96e29d admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.073104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab52d8-ae11-4c1f-9db1-99bdcf96e29d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.073381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab52d8-ae11-4c1f-9db1-99bdcf96e29d admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.074705 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:11.074901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab52d8-ae11-4c1f-9db1-99bdcf96e29d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:11.075105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab52d8-ae11-4c1f-9db1-99bdcf96e29d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.076711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab52d8-ae11-4c1f-9db1-99bdcf96e29d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.079917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955c674-13e6-4177-af03-682bf880f78e admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.084628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd21f23-00ee-4ac4-b322-3600c6dcf998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.085245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd21f23-00ee-4ac4-b322-3600c6dcf998 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.085524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd21f23-00ee-4ac4-b322-3600c6dcf998 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.085799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd21f23-00ee-4ac4-b322-3600c6dcf998 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.086513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd21f23-00ee-4ac4-b322-3600c6dcf998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:11.086819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd21f23-00ee-4ac4-b322-3600c6dcf998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.088393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd21f23-00ee-4ac4-b322-3600c6dcf998 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.088988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 783/1563] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:11.090483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab52d8-ae11-4c1f-9db1-99bdcf96e29d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.101259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 781/1564] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:11.102109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd21f23-00ee-4ac4-b322-3600c6dcf998 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.110217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab52d8-ae11-4c1f-9db1-99bdcf96e29d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.112527 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd21f23-00ee-4ac4-b322-3600c6dcf998 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.114941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9519e-a004-42c1-a28e-dde98145ff6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.115553 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9519e-a004-42c1-a28e-dde98145ff6f admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.116114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9519e-a004-42c1-a28e-dde98145ff6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.116395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9519e-a004-42c1-a28e-dde98145ff6f admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.117111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9519e-a004-42c1-a28e-dde98145ff6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.117477 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac51f4-39f9-43e8-9ce4-b24d250381af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.118101 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac51f4-39f9-43e8-9ce4-b24d250381af admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.118382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac51f4-39f9-43e8-9ce4-b24d250381af admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.118653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac51f4-39f9-43e8-9ce4-b24d250381af admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.118758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9519e-a004-42c1-a28e-dde98145ff6f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.119563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac51f4-39f9-43e8-9ce4-b24d250381af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:11.119709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac51f4-39f9-43e8-9ce4-b24d250381af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.122818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac51f4-39f9-43e8-9ce4-b24d250381af admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.129415 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 784/1565] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:11.130246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9519e-a004-42c1-a28e-dde98145ff6f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.135371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 782/1566] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:11.137184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac51f4-39f9-43e8-9ce4-b24d250381af admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.140300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9519e-a004-42c1-a28e-dde98145ff6f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.146342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb529ea-0f0d-4d64-bb32-b85ed2018532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.147508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb529ea-0f0d-4d64-bb32-b85ed2018532 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.147508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb529ea-0f0d-4d64-bb32-b85ed2018532 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.147707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb529ea-0f0d-4d64-bb32-b85ed2018532 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.148245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb529ea-0f0d-4d64-bb32-b85ed2018532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.149269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: WARNING py.warnings [None req-9eb529ea-0f0d-4d64-bb32-b85ed2018532 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:40:11.149269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: warnings.warn(msg) Aug 04 21:40:11.149269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: Aug 04 21:40:11.150489 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb529ea-0f0d-4d64-bb32-b85ed2018532 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.151050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac51f4-39f9-43e8-9ce4-b24d250381af admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.155871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c2058-8912-433e-b22a-dd98c84b077c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.156474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c2058-8912-433e-b22a-dd98c84b077c admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.156753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c2058-8912-433e-b22a-dd98c84b077c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.157025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c2058-8912-433e-b22a-dd98c84b077c admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.157707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c2058-8912-433e-b22a-dd98c84b077c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:11.158027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c2058-8912-433e-b22a-dd98c84b077c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.159652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c2058-8912-433e-b22a-dd98c84b077c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.172265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 783/1567] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:11.173395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c2058-8912-433e-b22a-dd98c84b077c admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.182768 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c2058-8912-433e-b22a-dd98c84b077c admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.188116 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85313ae1-db4b-466e-b5d1-a2636593ba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.197523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85313ae1-db4b-466e-b5d1-a2636593ba3a admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.197523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85313ae1-db4b-466e-b5d1-a2636593ba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.197523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85313ae1-db4b-466e-b5d1-a2636593ba3a admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.197523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85313ae1-db4b-466e-b5d1-a2636593ba3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.197523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85313ae1-db4b-466e-b5d1-a2636593ba3a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.274342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 785/1568] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:11 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:11.275157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb529ea-0f0d-4d64-bb32-b85ed2018532 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.286717 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 784/1569] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:11 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:11.287596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85313ae1-db4b-466e-b5d1-a2636593ba3a admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.339843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb529ea-0f0d-4d64-bb32-b85ed2018532 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.350827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85313ae1-db4b-466e-b5d1-a2636593ba3a admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.355823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d30f8b-9d94-4b80-91da-7bdc2242e757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.356445 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d30f8b-9d94-4b80-91da-7bdc2242e757 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.356723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d30f8b-9d94-4b80-91da-7bdc2242e757 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.357004 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d30f8b-9d94-4b80-91da-7bdc2242e757 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.357677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d30f8b-9d94-4b80-91da-7bdc2242e757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.359620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d30f8b-9d94-4b80-91da-7bdc2242e757 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.396016 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83be587-639c-42f4-a591-29e0e5840a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.396631 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83be587-639c-42f4-a591-29e0e5840a37 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.396915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83be587-639c-42f4-a591-29e0e5840a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.397188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83be587-639c-42f4-a591-29e0e5840a37 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.397851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83be587-639c-42f4-a591-29e0e5840a37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.399800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83be587-639c-42f4-a591-29e0e5840a37 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.442421 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 785/1570] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:11 2022] POST /identity/v3/projects => generated 393 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:11.443205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d30f8b-9d94-4b80-91da-7bdc2242e757 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.507162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 786/1571] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:11 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:11.507472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d30f8b-9d94-4b80-91da-7bdc2242e757 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.507577 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83be587-639c-42f4-a591-29e0e5840a37 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.508861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc4aaf-53da-4a02-9ea7-9162c74e1594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.509474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc4aaf-53da-4a02-9ea7-9162c74e1594 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.509821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc4aaf-53da-4a02-9ea7-9162c74e1594 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.510037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc4aaf-53da-4a02-9ea7-9162c74e1594 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.510702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc4aaf-53da-4a02-9ea7-9162c74e1594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.512826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc4aaf-53da-4a02-9ea7-9162c74e1594 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.515036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83be587-639c-42f4-a591-29e0e5840a37 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.521422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3692be-4ac5-43de-9280-211d3b56b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.521422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3692be-4ac5-43de-9280-211d3b56b223 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.521422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3692be-4ac5-43de-9280-211d3b56b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.521422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3692be-4ac5-43de-9280-211d3b56b223 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.524927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3692be-4ac5-43de-9280-211d3b56b223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.524927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3692be-4ac5-43de-9280-211d3b56b223 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.534662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 787/1572] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:11.535475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3692be-4ac5-43de-9280-211d3b56b223 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.545397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3692be-4ac5-43de-9280-211d3b56b223 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.549903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d5b7d-4d7c-403f-accb-e7478f8353c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.555353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d5b7d-4d7c-403f-accb-e7478f8353c5 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.555353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d5b7d-4d7c-403f-accb-e7478f8353c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.555353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d5b7d-4d7c-403f-accb-e7478f8353c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.555353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d5b7d-4d7c-403f-accb-e7478f8353c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.555353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d5b7d-4d7c-403f-accb-e7478f8353c5 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.606473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 786/1573] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:11 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:11.608132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc4aaf-53da-4a02-9ea7-9162c74e1594 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.621369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:11.638539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 788/1574] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:11 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:11.638539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d5b7d-4d7c-403f-accb-e7478f8353c5 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.657311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:11.671306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc4aaf-53da-4a02-9ea7-9162c74e1594 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.701125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d5b7d-4d7c-403f-accb-e7478f8353c5 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.705990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e56610-8600-489c-9fab-8dbacdd03b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.706859 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56610-8600-489c-9fab-8dbacdd03b35 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.707152 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56610-8600-489c-9fab-8dbacdd03b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.707429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56610-8600-489c-9fab-8dbacdd03b35 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.708111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e56610-8600-489c-9fab-8dbacdd03b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.710221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e56610-8600-489c-9fab-8dbacdd03b35 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.727278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7d313-1964-48f4-a371-f829c3871972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.727904 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7d313-1964-48f4-a371-f829c3871972 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.728188 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7d313-1964-48f4-a371-f829c3871972 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.728495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7d313-1964-48f4-a371-f829c3871972 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.729175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7d313-1964-48f4-a371-f829c3871972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.731751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7d313-1964-48f4-a371-f829c3871972 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.821681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 789/1575] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:11 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:11.821681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e56610-8600-489c-9fab-8dbacdd03b35 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.832542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e56610-8600-489c-9fab-8dbacdd03b35 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.837744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ce82d-0970-4ffb-bd52-50c60e8bea01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.838361 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ce82d-0970-4ffb-bd52-50c60e8bea01 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.838644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ce82d-0970-4ffb-bd52-50c60e8bea01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.838920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ce82d-0970-4ffb-bd52-50c60e8bea01 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.839709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ce82d-0970-4ffb-bd52-50c60e8bea01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.841755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ce82d-0970-4ffb-bd52-50c60e8bea01 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.848246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 787/1576] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:11 2022] POST /identity/v3/projects => generated 377 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:11.848246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7d313-1964-48f4-a371-f829c3871972 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.906879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7d313-1964-48f4-a371-f829c3871972 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.911550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e29130-5895-4f54-aac2-d604ebdab9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.912170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e29130-5895-4f54-aac2-d604ebdab9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.912454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e29130-5895-4f54-aac2-d604ebdab9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.912730 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e29130-5895-4f54-aac2-d604ebdab9e3 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.913424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e29130-5895-4f54-aac2-d604ebdab9e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.914501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: WARNING py.warnings [None req-d0e29130-5895-4f54-aac2-d604ebdab9e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:40:11.914501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: warnings.warn(msg) Aug 04 21:40:11.914501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: Aug 04 21:40:11.915727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e29130-5895-4f54-aac2-d604ebdab9e3 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.934074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 790/1577] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:11 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:11.934921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ce82d-0970-4ffb-bd52-50c60e8bea01 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.944185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ce82d-0970-4ffb-bd52-50c60e8bea01 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.949270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739dd4d9-4e31-4ab6-b6b3-bb21539f49c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.949849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739dd4d9-4e31-4ab6-b6b3-bb21539f49c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.950179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739dd4d9-4e31-4ab6-b6b3-bb21539f49c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.950457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739dd4d9-4e31-4ab6-b6b3-bb21539f49c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.951185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739dd4d9-4e31-4ab6-b6b3-bb21539f49c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.952714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739dd4d9-4e31-4ab6-b6b3-bb21539f49c0 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:11.962924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 791/1578] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:11.963664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739dd4d9-4e31-4ab6-b6b3-bb21539f49c0 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:11.973398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739dd4d9-4e31-4ab6-b6b3-bb21539f49c0 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:11.977870 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f652c-8beb-4c64-9f9f-d2c639b6d072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:11.978519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f652c-8beb-4c64-9f9f-d2c639b6d072 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:11.978802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f652c-8beb-4c64-9f9f-d2c639b6d072 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:11.979076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f652c-8beb-4c64-9f9f-d2c639b6d072 admin admin] PATH_INFO: `/v3/projects/0cdb0d7348374f9aba8720777b847b23/users/7c3d7bae0f004eac9bfe3770e2425091/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:11.993693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f652c-8beb-4c64-9f9f-d2c639b6d072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cdb0d7348374f9aba8720777b847b23, user_id=7c3d7bae0f004eac9bfe3770e2425091, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:11.994730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: WARNING py.warnings [None req-443f652c-8beb-4c64-9f9f-d2c639b6d072 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:40:11.994730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: warnings.warn(msg) Aug 04 21:40:11.994730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: Aug 04 21:40:11.996806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f652c-8beb-4c64-9f9f-d2c639b6d072 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.001353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 788/1579] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:11 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:12.002256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e29130-5895-4f54-aac2-d604ebdab9e3 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.011997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e29130-5895-4f54-aac2-d604ebdab9e3 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.016673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e06e-7bc4-4c1f-9769-ae1869df2b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.017249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e06e-7bc4-4c1f-9769-ae1869df2b49 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.017529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e06e-7bc4-4c1f-9769-ae1869df2b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.017805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e06e-7bc4-4c1f-9769-ae1869df2b49 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.018482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67e06e-7bc4-4c1f-9769-ae1869df2b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.020368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67e06e-7bc4-4c1f-9769-ae1869df2b49 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.069940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 792/1580] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:11 2022] PUT /identity/v3/projects/0cdb0d7348374f9aba8720777b847b23/users/7c3d7bae0f004eac9bfe3770e2425091/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:12.070789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f652c-8beb-4c64-9f9f-d2c639b6d072 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.098744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 789/1581] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:12 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:12.099575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e06e-7bc4-4c1f-9769-ae1869df2b49 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.109787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e06e-7bc4-4c1f-9769-ae1869df2b49 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.121905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:12.135936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f652c-8beb-4c64-9f9f-d2c639b6d072 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.140592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35450e48-b86b-46f9-b713-56fb94997117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.141205 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35450e48-b86b-46f9-b713-56fb94997117 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.141486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35450e48-b86b-46f9-b713-56fb94997117 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.141780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35450e48-b86b-46f9-b713-56fb94997117 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.142512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35450e48-b86b-46f9-b713-56fb94997117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.144438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35450e48-b86b-46f9-b713-56fb94997117 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.169235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c8492-5e64-4fe2-913b-4d402eefc9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.170554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c8492-5e64-4fe2-913b-4d402eefc9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.170554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c8492-5e64-4fe2-913b-4d402eefc9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.170554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c8492-5e64-4fe2-913b-4d402eefc9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.171257 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c8492-5e64-4fe2-913b-4d402eefc9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.172863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c8492-5e64-4fe2-913b-4d402eefc9d7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.182521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 790/1582] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.183464 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c8492-5e64-4fe2-913b-4d402eefc9d7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.193122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c8492-5e64-4fe2-913b-4d402eefc9d7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.197830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cfead3-14de-4ff1-8864-475b6fa6faf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.199156 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfead3-14de-4ff1-8864-475b6fa6faf9 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.199328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfead3-14de-4ff1-8864-475b6fa6faf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.199557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfead3-14de-4ff1-8864-475b6fa6faf9 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.200194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cfead3-14de-4ff1-8864-475b6fa6faf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.202112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cfead3-14de-4ff1-8864-475b6fa6faf9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.255667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 793/1583] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:12 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:12.256488 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35450e48-b86b-46f9-b713-56fb94997117 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.266405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35450e48-b86b-46f9-b713-56fb94997117 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.270950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a772f27-3134-4ef7-99ac-653689e84569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.271528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772f27-3134-4ef7-99ac-653689e84569 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.271808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772f27-3134-4ef7-99ac-653689e84569 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.272079 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772f27-3134-4ef7-99ac-653689e84569 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.272754 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a772f27-3134-4ef7-99ac-653689e84569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.274403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a772f27-3134-4ef7-99ac-653689e84569 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.284916 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 794/1584] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.285727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a772f27-3134-4ef7-99ac-653689e84569 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.295001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a772f27-3134-4ef7-99ac-653689e84569 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.299525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548c571-2979-410e-8b8c-788ead2e7ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.300094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548c571-2979-410e-8b8c-788ead2e7ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.300376 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548c571-2979-410e-8b8c-788ead2e7ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.300647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548c571-2979-410e-8b8c-788ead2e7ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.301317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548c571-2979-410e-8b8c-788ead2e7ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.302897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548c571-2979-410e-8b8c-788ead2e7ab0 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.312971 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 795/1585] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.314949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548c571-2979-410e-8b8c-788ead2e7ab0 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.322257 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 791/1586] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:12 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:12.322845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548c571-2979-410e-8b8c-788ead2e7ab0 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.324113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f1dab-628e-4260-928d-cf2ce115e284 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.324463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f1dab-628e-4260-928d-cf2ce115e284 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.324741 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f1dab-628e-4260-928d-cf2ce115e284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.329744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631ecc4-ea5d-441b-8d49-ea5e325e939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.330381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631ecc4-ea5d-441b-8d49-ea5e325e939a admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.330672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631ecc4-ea5d-441b-8d49-ea5e325e939a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.330936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631ecc4-ea5d-441b-8d49-ea5e325e939a admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.331624 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631ecc4-ea5d-441b-8d49-ea5e325e939a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.333182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631ecc4-ea5d-441b-8d49-ea5e325e939a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.344434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-dc1f1dab-628e-4260-928d-cf2ce115e284 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:12.345069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 796/1587] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.345743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-dc1f1dab-628e-4260-928d-cf2ce115e284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:12.346044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631ecc4-ea5d-441b-8d49-ea5e325e939a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.353912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 792/1588] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:12.354749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f1dab-628e-4260-928d-cf2ce115e284 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.356028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631ecc4-ea5d-441b-8d49-ea5e325e939a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.363607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196c960-5660-422d-a105-dc9ad19aca25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.364229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c960-5660-422d-a105-dc9ad19aca25 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.364345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f1dab-628e-4260-928d-cf2ce115e284 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.364518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c960-5660-422d-a105-dc9ad19aca25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.364799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c960-5660-422d-a105-dc9ad19aca25 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.365493 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196c960-5660-422d-a105-dc9ad19aca25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.367479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196c960-5660-422d-a105-dc9ad19aca25 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.369157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8c321-758d-4a5c-ab87-df900f5aea8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.369967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8c321-758d-4a5c-ab87-df900f5aea8e admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.370105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:12.370213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8c321-758d-4a5c-ab87-df900f5aea8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.370360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8c321-758d-4a5c-ab87-df900f5aea8e admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.371050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8c321-758d-4a5c-ab87-df900f5aea8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.372646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8c321-758d-4a5c-ab87-df900f5aea8e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.380000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 797/1589] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.385541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196c960-5660-422d-a105-dc9ad19aca25 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.385660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 793/1590] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.385660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8c321-758d-4a5c-ab87-df900f5aea8e admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.391810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196c960-5660-422d-a105-dc9ad19aca25 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.394607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8c321-758d-4a5c-ab87-df900f5aea8e admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.396713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3cc6b-b5e4-4eb7-b23e-ac33baaac768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.397517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3cc6b-b5e4-4eb7-b23e-ac33baaac768 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.397639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3cc6b-b5e4-4eb7-b23e-ac33baaac768 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.397861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3cc6b-b5e4-4eb7-b23e-ac33baaac768 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.398602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3cc6b-b5e4-4eb7-b23e-ac33baaac768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.399526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c100a-6f91-45d6-a06e-cd1682135819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.400115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c100a-6f91-45d6-a06e-cd1682135819 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.400255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3cc6b-b5e4-4eb7-b23e-ac33baaac768 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.400421 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c100a-6f91-45d6-a06e-cd1682135819 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.400654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c100a-6f91-45d6-a06e-cd1682135819 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.401327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c100a-6f91-45d6-a06e-cd1682135819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.403515 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c100a-6f91-45d6-a06e-cd1682135819 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.414497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 794/1591] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.418370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 798/1592] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.418482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c100a-6f91-45d6-a06e-cd1682135819 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.418578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3cc6b-b5e4-4eb7-b23e-ac33baaac768 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.423820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c100a-6f91-45d6-a06e-cd1682135819 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.424371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3cc6b-b5e4-4eb7-b23e-ac33baaac768 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.428896 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bc657-c086-4a71-8520-08012d073b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.429044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686be48-e968-4ac5-aa50-3859905203ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.429507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bc657-c086-4a71-8520-08012d073b40 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.429619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686be48-e968-4ac5-aa50-3859905203ee admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.429789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bc657-c086-4a71-8520-08012d073b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.429923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686be48-e968-4ac5-aa50-3859905203ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.430098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bc657-c086-4a71-8520-08012d073b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.430203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686be48-e968-4ac5-aa50-3859905203ee admin admin] PATH_INFO: `/v3/projects/09370df0a2354bd2a412c6770d3ff165/users/a23fcabad2354e7ab51a22b412bf11e6/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.430785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bc657-c086-4a71-8520-08012d073b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.432384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bc657-c086-4a71-8520-08012d073b40 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.442711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 799/1593] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.443588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bc657-c086-4a71-8520-08012d073b40 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.448177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686be48-e968-4ac5-aa50-3859905203ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=09370df0a2354bd2a412c6770d3ff165, user_id=a23fcabad2354e7ab51a22b412bf11e6, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.451371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686be48-e968-4ac5-aa50-3859905203ee admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.453708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bc657-c086-4a71-8520-08012d073b40 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.458427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ab841-add3-4b98-8161-06a6f06c5eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.459051 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ab841-add3-4b98-8161-06a6f06c5eab admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.459334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ab841-add3-4b98-8161-06a6f06c5eab admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.459605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ab841-add3-4b98-8161-06a6f06c5eab admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.460308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ab841-add3-4b98-8161-06a6f06c5eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.461867 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ab841-add3-4b98-8161-06a6f06c5eab admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.472316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 800/1594] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.473105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ab841-add3-4b98-8161-06a6f06c5eab admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.531176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 795/1595] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:12 2022] PUT /identity/v3/projects/09370df0a2354bd2a412c6770d3ff165/users/a23fcabad2354e7ab51a22b412bf11e6/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:12.531365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ab841-add3-4b98-8161-06a6f06c5eab admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.531476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686be48-e968-4ac5-aa50-3859905203ee admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.576906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da56196-4eb0-4992-b02b-7744c749a0dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.577521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56196-4eb0-4992-b02b-7744c749a0dc service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.577801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56196-4eb0-4992-b02b-7744c749a0dc service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.578103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56196-4eb0-4992-b02b-7744c749a0dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.591753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686be48-e968-4ac5-aa50-3859905203ee admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.596925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840dacf5-ddcb-4b51-9e7a-5e82cf516fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.597923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dacf5-ddcb-4b51-9e7a-5e82cf516fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.598237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dacf5-ddcb-4b51-9e7a-5e82cf516fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.598513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dacf5-ddcb-4b51-9e7a-5e82cf516fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.599208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840dacf5-ddcb-4b51-9e7a-5e82cf516fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.600809 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840dacf5-ddcb-4b51-9e7a-5e82cf516fc0 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.611799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 796/1596] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.612643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840dacf5-ddcb-4b51-9e7a-5e82cf516fc0 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.622673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840dacf5-ddcb-4b51-9e7a-5e82cf516fc0 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.627460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffba1ef-25ec-4718-91c8-2cef8e4c29da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.628112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffba1ef-25ec-4718-91c8-2cef8e4c29da admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.628393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffba1ef-25ec-4718-91c8-2cef8e4c29da admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.628668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffba1ef-25ec-4718-91c8-2cef8e4c29da admin admin] PATH_INFO: `/v3/projects/7b6134537ca84005a94c5e82b11277fd/users/9b916c33b04e494eb413f6832379bf0c/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.643044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffba1ef-25ec-4718-91c8-2cef8e4c29da admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6134537ca84005a94c5e82b11277fd, user_id=9b916c33b04e494eb413f6832379bf0c, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.644215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da56196-4eb0-4992-b02b-7744c749a0dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.645636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da56196-4eb0-4992-b02b-7744c749a0dc service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.645792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffba1ef-25ec-4718-91c8-2cef8e4c29da admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.655893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 801/1597] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:12.656684 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da56196-4eb0-4992-b02b-7744c749a0dc service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.665770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da56196-4eb0-4992-b02b-7744c749a0dc service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.671785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9008b-94ee-47fd-8ef4-e9b8945372a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.672503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9008b-94ee-47fd-8ef4-e9b8945372a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.672753 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9008b-94ee-47fd-8ef4-e9b8945372a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.673025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9008b-94ee-47fd-8ef4-e9b8945372a3 admin admin] PATH_INFO: `/v3/projects/df5356c0de064c2e83109c058870efb1/users/cd348c0520e246d5814e035349bf7b90/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.688013 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9008b-94ee-47fd-8ef4-e9b8945372a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5356c0de064c2e83109c058870efb1, user_id=cd348c0520e246d5814e035349bf7b90, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.690685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9008b-94ee-47fd-8ef4-e9b8945372a3 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.724086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 797/1598] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:12 2022] PUT /identity/v3/projects/7b6134537ca84005a94c5e82b11277fd/users/9b916c33b04e494eb413f6832379bf0c/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:12.724086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffba1ef-25ec-4718-91c8-2cef8e4c29da admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.741907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:12.788848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffba1ef-25ec-4718-91c8-2cef8e4c29da admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.794195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c4135-516b-485c-b49f-d172cc26c6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.794808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c4135-516b-485c-b49f-d172cc26c6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.795087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c4135-516b-485c-b49f-d172cc26c6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.795359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c4135-516b-485c-b49f-d172cc26c6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.796040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c4135-516b-485c-b49f-d172cc26c6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.797629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c4135-516b-485c-b49f-d172cc26c6a8 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.807825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 802/1599] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:12 2022] PUT /identity/v3/projects/df5356c0de064c2e83109c058870efb1/users/cd348c0520e246d5814e035349bf7b90/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:12.808917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9008b-94ee-47fd-8ef4-e9b8945372a3 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.811163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 798/1600] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.816624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c4135-516b-485c-b49f-d172cc26c6a8 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.819585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:12.827991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c4135-516b-485c-b49f-d172cc26c6a8 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.881734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9008b-94ee-47fd-8ef4-e9b8945372a3 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.886686 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc96e3f-5448-4b9f-9e3a-5a3d9627542e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.887306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc96e3f-5448-4b9f-9e3a-5a3d9627542e admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.887587 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc96e3f-5448-4b9f-9e3a-5a3d9627542e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.887859 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc96e3f-5448-4b9f-9e3a-5a3d9627542e admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.888550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc96e3f-5448-4b9f-9e3a-5a3d9627542e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.890177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc96e3f-5448-4b9f-9e3a-5a3d9627542e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.895483 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeaa47f-47e0-453a-9b8a-9adc8cbf4f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.896152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaa47f-47e0-453a-9b8a-9adc8cbf4f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.896435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaa47f-47e0-453a-9b8a-9adc8cbf4f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.896711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaa47f-47e0-453a-9b8a-9adc8cbf4f3b admin admin] PATH_INFO: `/v3/projects/26fb895147494be98bb6ceb8704984b7/users/d9bc42bec393430b9b3e9399a8c4f95e/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.900222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 803/1601] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:12.905460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b48c8-bbaa-4447-8bfd-1ab6135c94fc None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.905756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b48c8-bbaa-4447-8bfd-1ab6135c94fc None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.906047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b48c8-bbaa-4447-8bfd-1ab6135c94fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.914266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeaa47f-47e0-453a-9b8a-9adc8cbf4f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=26fb895147494be98bb6ceb8704984b7, user_id=d9bc42bec393430b9b3e9399a8c4f95e, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.916293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeaa47f-47e0-453a-9b8a-9adc8cbf4f3b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:12.925462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-289b48c8-bbaa-4447-8bfd-1ab6135c94fc None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:12.937185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-289b48c8-bbaa-4447-8bfd-1ab6135c94fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:12.937185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 804/1602] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:12.937185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b48c8-bbaa-4447-8bfd-1ab6135c94fc None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:12.944411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b48c8-bbaa-4447-8bfd-1ab6135c94fc None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:12.949485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ccbd5-0076-4305-a68b-8c925f684ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:12.950165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ccbd5-0076-4305-a68b-8c925f684ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:12.950453 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ccbd5-0076-4305-a68b-8c925f684ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:12.950709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ccbd5-0076-4305-a68b-8c925f684ce3 admin admin] PATH_INFO: `/v3/projects/4b95ce24a4d34e379fa86c9114ece400/users/95fb2edf82644e7f8ce461fcbb52007b/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:12.965740 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ccbd5-0076-4305-a68b-8c925f684ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b95ce24a4d34e379fa86c9114ece400, user_id=95fb2edf82644e7f8ce461fcbb52007b, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:12.968502 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ccbd5-0076-4305-a68b-8c925f684ce3 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:13.022343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 799/1603] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:12 2022] PUT /identity/v3/projects/26fb895147494be98bb6ceb8704984b7/users/d9bc42bec393430b9b3e9399a8c4f95e/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:13.022343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf8781-88de-4078-8906-e4cbf26363a6 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.022343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf8781-88de-4078-8906-e4cbf26363a6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.022343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf8781-88de-4078-8906-e4cbf26363a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.052010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-97cf8781-88de-4078-8906-e4cbf26363a6 None None] MFA Rules not processed for user `9b916c33b04e494eb413f6832379bf0c`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:13.053229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-97cf8781-88de-4078-8906-e4cbf26363a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:13.077605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 805/1604] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:12 2022] PUT /identity/v3/projects/4b95ce24a4d34e379fa86c9114ece400/users/95fb2edf82644e7f8ce461fcbb52007b/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:13.078765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ccbd5-0076-4305-a68b-8c925f684ce3 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:13.131780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 800/1605] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:13.133267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fcbeb-8043-4a35-a759-0d4a1ead49b6 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.133620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fcbeb-8043-4a35-a759-0d4a1ead49b6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.133964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fcbeb-8043-4a35-a759-0d4a1ead49b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.153565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ccbd5-0076-4305-a68b-8c925f684ce3 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:13.158273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684cdb7-535b-42ce-bd18-c7a9f7a14a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:13.158933 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684cdb7-535b-42ce-bd18-c7a9f7a14a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.159213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684cdb7-535b-42ce-bd18-c7a9f7a14a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.159487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684cdb7-535b-42ce-bd18-c7a9f7a14a49 admin admin] PATH_INFO: `/v3/projects/138cb60729014ec599089a01fba5553f/users/1d016173354445d699d0007610ee0144/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.165670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-779fcbeb-8043-4a35-a759-0d4a1ead49b6 None None] MFA Rules not processed for user `cd348c0520e246d5814e035349bf7b90`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:13.166917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-779fcbeb-8043-4a35-a759-0d4a1ead49b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:13.178242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:13.183905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684cdb7-535b-42ce-bd18-c7a9f7a14a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=138cb60729014ec599089a01fba5553f, user_id=1d016173354445d699d0007610ee0144, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:13.187144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684cdb7-535b-42ce-bd18-c7a9f7a14a49 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:13.235047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 801/1606] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:13.235796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fcbeb-8043-4a35-a759-0d4a1ead49b6 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:13.249392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fcbeb-8043-4a35-a759-0d4a1ead49b6 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:13.254113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76127989-c5da-4f21-abfc-24dcd16bb967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:13.254840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76127989-c5da-4f21-abfc-24dcd16bb967 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.255930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76127989-c5da-4f21-abfc-24dcd16bb967 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.255930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76127989-c5da-4f21-abfc-24dcd16bb967 admin admin] PATH_INFO: `/v3/projects/4c113838ba92441c89e4447d7d567b31/users/d681a6cc3e5247c9bb2537a6cacb7dd4/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.275374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76127989-c5da-4f21-abfc-24dcd16bb967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c113838ba92441c89e4447d7d567b31, user_id=d681a6cc3e5247c9bb2537a6cacb7dd4, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:13.279115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76127989-c5da-4f21-abfc-24dcd16bb967 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:13.302370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 806/1607] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:13 2022] PUT /identity/v3/projects/138cb60729014ec599089a01fba5553f/users/1d016173354445d699d0007610ee0144/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:13.307765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684cdb7-535b-42ce-bd18-c7a9f7a14a49 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:13.309984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0684cdb7-535b-42ce-bd18-c7a9f7a14a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:13.371846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684cdb7-535b-42ce-bd18-c7a9f7a14a49 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:13.375456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de047d30-cc6d-423e-8c40-b35fe35c7069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:13.376069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de047d30-cc6d-423e-8c40-b35fe35c7069 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.376497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de047d30-cc6d-423e-8c40-b35fe35c7069 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.376670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de047d30-cc6d-423e-8c40-b35fe35c7069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.384283 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 802/1608] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:13 2022] PUT /identity/v3/projects/4c113838ba92441c89e4447d7d567b31/users/d681a6cc3e5247c9bb2537a6cacb7dd4/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:13.388055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:13.388189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debab8ea-1cfc-47db-907e-96fb776d16f6 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.388189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debab8ea-1cfc-47db-907e-96fb776d16f6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.388189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debab8ea-1cfc-47db-907e-96fb776d16f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.405785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-debab8ea-1cfc-47db-907e-96fb776d16f6 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:13.410846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-debab8ea-1cfc-47db-907e-96fb776d16f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:13.446552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de047d30-cc6d-423e-8c40-b35fe35c7069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:13.448072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de047d30-cc6d-423e-8c40-b35fe35c7069 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:13.459266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 807/1609] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:13.460370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec35ea-f3b8-4748-b613-3135480cf1e4 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.460493 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec35ea-f3b8-4748-b613-3135480cf1e4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.460700 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec35ea-f3b8-4748-b613-3135480cf1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.477675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 803/1610] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:13.478759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debab8ea-1cfc-47db-907e-96fb776d16f6 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:13.485458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a5ec35ea-f3b8-4748-b613-3135480cf1e4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:13.486765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a5ec35ea-f3b8-4748-b613-3135480cf1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:13.496518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 808/1611] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:13.501797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9add0-734c-4e72-b6c2-9cca9cf90806 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.501797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9add0-734c-4e72-b6c2-9cca9cf90806 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.501797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9add0-734c-4e72-b6c2-9cca9cf90806 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.501797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 809/1612] 10.222.0.73 () {54 vars in 962 bytes} [Thu Aug 4 21:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:40:13.501797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455882ab-22f1-48e9-a793-3364127f1312 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.501797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455882ab-22f1-48e9-a793-3364127f1312 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.502398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455882ab-22f1-48e9-a793-3364127f1312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.539666 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-455882ab-22f1-48e9-a793-3364127f1312 None None] MFA Rules not processed for user `1d016173354445d699d0007610ee0144`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:13.540871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-455882ab-22f1-48e9-a793-3364127f1312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:13.560288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debab8ea-1cfc-47db-907e-96fb776d16f6 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:13.564298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b6cc9-03f7-46fa-ad06-715b57fb5113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:13.564929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b6cc9-03f7-46fa-ad06-715b57fb5113 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.565194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b6cc9-03f7-46fa-ad06-715b57fb5113 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.565540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b6cc9-03f7-46fa-ad06-715b57fb5113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.611576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:13.622266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 810/1613] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:13.622266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823c540-e155-4417-9b9a-2dd5b5023d9c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.622266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823c540-e155-4417-9b9a-2dd5b5023d9c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.622266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823c540-e155-4417-9b9a-2dd5b5023d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.632305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b6cc9-03f7-46fa-ad06-715b57fb5113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:13.634285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b6cc9-03f7-46fa-ad06-715b57fb5113 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:13.640583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-e823c540-e155-4417-9b9a-2dd5b5023d9c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:13.641788 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e823c540-e155-4417-9b9a-2dd5b5023d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:13.644672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 804/1614] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:13.645629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b6cc9-03f7-46fa-ad06-715b57fb5113 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:13.650101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 811/1615] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:13.652210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c765c-6663-42c6-b725-4d99cc701171 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.652383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c765c-6663-42c6-b725-4d99cc701171 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.652628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c765c-6663-42c6-b725-4d99cc701171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.704410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ab0c765c-6663-42c6-b725-4d99cc701171 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:13.705913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ab0c765c-6663-42c6-b725-4d99cc701171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:13.719645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b6cc9-03f7-46fa-ad06-715b57fb5113 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:13.724875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3400d-ec4d-4b7a-937f-0166df397c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:13.725612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3400d-ec4d-4b7a-937f-0166df397c92 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.725886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3400d-ec4d-4b7a-937f-0166df397c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.726238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3400d-ec4d-4b7a-937f-0166df397c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.741443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3400d-ec4d-4b7a-937f-0166df397c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:13.742877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3400d-ec4d-4b7a-937f-0166df397c92 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:13.753200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 805/1616] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:13.754075 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3400d-ec4d-4b7a-937f-0166df397c92 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:13.765626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3400d-ec4d-4b7a-937f-0166df397c92 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:13.775308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c26a2e-a01d-4158-98e8-41da3f16208f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:13.776178 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c26a2e-a01d-4158-98e8-41da3f16208f service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.776895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c26a2e-a01d-4158-98e8-41da3f16208f service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.777341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c26a2e-a01d-4158-98e8-41da3f16208f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.796423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c26a2e-a01d-4158-98e8-41da3f16208f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:13.797863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c26a2e-a01d-4158-98e8-41da3f16208f service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:13.814214 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 806/1617] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:13.814214 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c26a2e-a01d-4158-98e8-41da3f16208f service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:13.822921 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c26a2e-a01d-4158-98e8-41da3f16208f service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:13.828366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0c898-16be-4aad-ab41-dd78424e0ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:13.829011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0c898-16be-4aad-ab41-dd78424e0ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.829339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0c898-16be-4aad-ab41-dd78424e0ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.829664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0c898-16be-4aad-ab41-dd78424e0ac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.830485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 812/1618] 10.222.0.73 () {58 vars in 1059 bytes} [Thu Aug 4 21:40:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:13.830603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0c898-16be-4aad-ab41-dd78424e0ac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:13.831839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c765c-6663-42c6-b725-4d99cc701171 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:13.833166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0c898-16be-4aad-ab41-dd78424e0ac7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:13.845400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c765c-6663-42c6-b725-4d99cc701171 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:13.850290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04946a0-6075-4691-80d6-60c68b74c23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:13.851087 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04946a0-6075-4691-80d6-60c68b74c23c service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.851834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04946a0-6075-4691-80d6-60c68b74c23c service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.852228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04946a0-6075-4691-80d6-60c68b74c23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.867318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04946a0-6075-4691-80d6-60c68b74c23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:13.872827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04946a0-6075-4691-80d6-60c68b74c23c service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:13.882524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 813/1619] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:13.883391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04946a0-6075-4691-80d6-60c68b74c23c service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:13.897816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04946a0-6075-4691-80d6-60c68b74c23c service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:13.905580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ebe31-2e68-4e4e-b865-f5d83d8cac81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:13.906449 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ebe31-2e68-4e4e-b865-f5d83d8cac81 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:13.907266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ebe31-2e68-4e4e-b865-f5d83d8cac81 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:13.907550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ebe31-2e68-4e4e-b865-f5d83d8cac81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:13.938349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 807/1620] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:13 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:13.938572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:13.939890 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0c898-16be-4aad-ab41-dd78424e0ac7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:14.035489 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ebe31-2e68-4e4e-b865-f5d83d8cac81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:14.038654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ebe31-2e68-4e4e-b865-f5d83d8cac81 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:14.049593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0c898-16be-4aad-ab41-dd78424e0ac7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:14.052770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 814/1621] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:14.055124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62633e-324b-4299-ae2c-80836047396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:14.055746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62633e-324b-4299-ae2c-80836047396f admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:14.056395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62633e-324b-4299-ae2c-80836047396f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:14.056697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62633e-324b-4299-ae2c-80836047396f admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:14.057354 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62633e-324b-4299-ae2c-80836047396f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:14.059622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62633e-324b-4299-ae2c-80836047396f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:14.185393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 808/1622] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:13 2022] POST /identity/v3/users => generated 292 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:14.188054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ebe31-2e68-4e4e-b865-f5d83d8cac81 service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:14.198965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ebe31-2e68-4e4e-b865-f5d83d8cac81 service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:14.204017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa045d3-668d-4cee-9c66-c9b91b64610d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:14.205138 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa045d3-668d-4cee-9c66-c9b91b64610d admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:14.205138 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa045d3-668d-4cee-9c66-c9b91b64610d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:14.211661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa045d3-668d-4cee-9c66-c9b91b64610d admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:14.211661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa045d3-668d-4cee-9c66-c9b91b64610d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:14.211661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa045d3-668d-4cee-9c66-c9b91b64610d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:14.232852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 815/1623] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:14.236434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62633e-324b-4299-ae2c-80836047396f admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:14.248543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62633e-324b-4299-ae2c-80836047396f admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:14.253990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319b61c-b56c-4c0b-8f9b-bfa203ac3f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:14.255243 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319b61c-b56c-4c0b-8f9b-bfa203ac3f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:14.255535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319b61c-b56c-4c0b-8f9b-bfa203ac3f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:14.255814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319b61c-b56c-4c0b-8f9b-bfa203ac3f8f admin admin] PATH_INFO: `/v3/projects/e53decc6afec43cd8e1e5f74d6f0f858/users/6193dd552d6e4194b7e9f41103c865b6/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:14.276627 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319b61c-b56c-4c0b-8f9b-bfa203ac3f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53decc6afec43cd8e1e5f74d6f0f858, user_id=6193dd552d6e4194b7e9f41103c865b6, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:14.281612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319b61c-b56c-4c0b-8f9b-bfa203ac3f8f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:14.314511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e4e51-6913-4959-96d2-fa1e3564e8bf None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:14.314511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e4e51-6913-4959-96d2-fa1e3564e8bf None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:14.314511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e4e51-6913-4959-96d2-fa1e3564e8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:14.368210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-022e4e51-6913-4959-96d2-fa1e3564e8bf None None] MFA Rules not processed for user `7c3d7bae0f004eac9bfe3770e2425091`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:14.369708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-022e4e51-6913-4959-96d2-fa1e3564e8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:14.403872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 809/1624] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:14 2022] PUT /identity/v3/projects/e53decc6afec43cd8e1e5f74d6f0f858/users/6193dd552d6e4194b7e9f41103c865b6/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:14.403872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07033a38-d835-42d3-bf04-57bcf5d31db7 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:14.403872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07033a38-d835-42d3-bf04-57bcf5d31db7 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:14.404948 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07033a38-d835-42d3-bf04-57bcf5d31db7 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:14.407019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 810/1625] 10.222.0.73 () {54 vars in 912 bytes} [Thu Aug 4 21:40:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:40:14.410100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07033a38-d835-42d3-bf04-57bcf5d31db7 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:14.447370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:14.507576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07033a38-d835-42d3-bf04-57bcf5d31db7 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:14.514820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326d9d9-d52d-4fa0-820f-e7afacb03a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:14.515616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326d9d9-d52d-4fa0-820f-e7afacb03a62 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:14.516132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326d9d9-d52d-4fa0-820f-e7afacb03a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:14.516349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326d9d9-d52d-4fa0-820f-e7afacb03a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:14.517117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326d9d9-d52d-4fa0-820f-e7afacb03a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:14.518888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326d9d9-d52d-4fa0-820f-e7afacb03a62 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:14.525093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 816/1626] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:14 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:14.525731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124ebb9-0b77-4d04-b470-ea8d6a431103 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:14.528764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124ebb9-0b77-4d04-b470-ea8d6a431103 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:14.528764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124ebb9-0b77-4d04-b470-ea8d6a431103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:14.546205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 811/1627] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:14.547815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326d9d9-d52d-4fa0-820f-e7afacb03a62 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:14.554405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-e124ebb9-0b77-4d04-b470-ea8d6a431103 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:14.555341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e124ebb9-0b77-4d04-b470-ea8d6a431103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:14.560941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326d9d9-d52d-4fa0-820f-e7afacb03a62 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:14.569334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0d857-2704-4405-903c-9a33ce92df24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:14.570161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0d857-2704-4405-903c-9a33ce92df24 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:14.570532 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0d857-2704-4405-903c-9a33ce92df24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:14.571010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0d857-2704-4405-903c-9a33ce92df24 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:14.571976 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0d857-2704-4405-903c-9a33ce92df24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:14.578617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0d857-2704-4405-903c-9a33ce92df24 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:14.626240 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:14.641163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 817/1628] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:40:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:14.642184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124ebb9-0b77-4d04-b470-ea8d6a431103 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:14.659137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124ebb9-0b77-4d04-b470-ea8d6a431103 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:14.664297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46410c-3ec9-4979-be40-01b4ef25fb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:14.664857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46410c-3ec9-4979-be40-01b4ef25fb7e admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:14.665153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46410c-3ec9-4979-be40-01b4ef25fb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:14.665432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46410c-3ec9-4979-be40-01b4ef25fb7e admin admin] PATH_INFO: `/v3/projects/e53decc6afec43cd8e1e5f74d6f0f858/users/6193dd552d6e4194b7e9f41103c865b6/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:14.667642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46410c-3ec9-4979-be40-01b4ef25fb7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53decc6afec43cd8e1e5f74d6f0f858, user_id=6193dd552d6e4194b7e9f41103c865b6, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:14.671245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46410c-3ec9-4979-be40-01b4ef25fb7e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:14.681419 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 818/1629] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:14 2022] PUT /identity/v3/projects/e53decc6afec43cd8e1e5f74d6f0f858/users/6193dd552d6e4194b7e9f41103c865b6/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:14.682654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46410c-3ec9-4979-be40-01b4ef25fb7e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:14.754767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46410c-3ec9-4979-be40-01b4ef25fb7e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:14.758438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131eb13b-6a72-451f-8a9f-18115d7415aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:14.764362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eb13b-6a72-451f-8a9f-18115d7415aa service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:14.764362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eb13b-6a72-451f-8a9f-18115d7415aa service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:14.764362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eb13b-6a72-451f-8a9f-18115d7415aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:14.764675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 812/1630] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:14 2022] POST /identity/v3/projects => generated 405 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:14.764675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3464-1a6e-4a62-a5a8-9f2caef2e0ed None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:14.764675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3464-1a6e-4a62-a5a8-9f2caef2e0ed None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:14.764675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3464-1a6e-4a62-a5a8-9f2caef2e0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:14.812867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-28ae3464-1a6e-4a62-a5a8-9f2caef2e0ed None None] MFA Rules not processed for user `6193dd552d6e4194b7e9f41103c865b6`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:14.814723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-28ae3464-1a6e-4a62-a5a8-9f2caef2e0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:14.878279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131eb13b-6a72-451f-8a9f-18115d7415aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:14.880111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131eb13b-6a72-451f-8a9f-18115d7415aa service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:14.898271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 819/1631] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:14.899504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131eb13b-6a72-451f-8a9f-18115d7415aa service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:14.941180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 813/1632] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:14.952932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae3464-1a6e-4a62-a5a8-9f2caef2e0ed None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:14.990135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131eb13b-6a72-451f-8a9f-18115d7415aa service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:15.055440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:15.060024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae3464-1a6e-4a62-a5a8-9f2caef2e0ed None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:15.070464 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18839459-f6ce-4848-a862-d53fe02dda82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:15.072244 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18839459-f6ce-4848-a862-d53fe02dda82 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:15.072449 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebddd5-0b96-43e4-a78d-85b9e13c36b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:15.073602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebddd5-0b96-43e4-a78d-85b9e13c36b7 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:15.073602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebddd5-0b96-43e4-a78d-85b9e13c36b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:15.073803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebddd5-0b96-43e4-a78d-85b9e13c36b7 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:15.074646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebddd5-0b96-43e4-a78d-85b9e13c36b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:15.076788 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebddd5-0b96-43e4-a78d-85b9e13c36b7 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:15.089562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18839459-f6ce-4848-a862-d53fe02dda82 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:15.090019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18839459-f6ce-4848-a862-d53fe02dda82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:15.117738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18839459-f6ce-4848-a862-d53fe02dda82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:15.119382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18839459-f6ce-4848-a862-d53fe02dda82 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:15.147771 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 814/1633] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:40:15.201983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 820/1634] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:14 2022] POST /identity/v3/users => generated 309 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:15.206208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18839459-f6ce-4848-a862-d53fe02dda82 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:15.224955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18839459-f6ce-4848-a862-d53fe02dda82 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:15.244624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d674b-12a3-4b62-b2e6-7398c45acbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:15.247365 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d674b-12a3-4b62-b2e6-7398c45acbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:15.247656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d674b-12a3-4b62-b2e6-7398c45acbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:15.247936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d674b-12a3-4b62-b2e6-7398c45acbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:15.251850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d674b-12a3-4b62-b2e6-7398c45acbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:15.253509 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d674b-12a3-4b62-b2e6-7398c45acbf2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:15.283811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 815/1635] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:15.289212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebddd5-0b96-43e4-a78d-85b9e13c36b7 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:15.304590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebddd5-0b96-43e4-a78d-85b9e13c36b7 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:15.317510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2b5de-88d4-4ae4-8f7a-5b49b1b6bccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:15.318370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b5de-88d4-4ae4-8f7a-5b49b1b6bccc admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:15.319466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b5de-88d4-4ae4-8f7a-5b49b1b6bccc admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:15.319603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b5de-88d4-4ae4-8f7a-5b49b1b6bccc admin admin] PATH_INFO: `/v3/projects/4d8645f9acf14d038ffacccc378eaf9f/users/538b2f4067b541d5a16a0b67caf0e73d/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:15.359060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2b5de-88d4-4ae4-8f7a-5b49b1b6bccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d8645f9acf14d038ffacccc378eaf9f, user_id=538b2f4067b541d5a16a0b67caf0e73d, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:15.364452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2b5de-88d4-4ae4-8f7a-5b49b1b6bccc admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:15.494728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 821/1636] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:15 2022] PUT /identity/v3/projects/4d8645f9acf14d038ffacccc378eaf9f/users/538b2f4067b541d5a16a0b67caf0e73d/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:15.499496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d674b-12a3-4b62-b2e6-7398c45acbf2 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:15.510510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d674b-12a3-4b62-b2e6-7398c45acbf2 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:15.525517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:15.601449 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d06d4-d1fd-4bf1-a92b-6a8c83034f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:15.602108 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d06d4-d1fd-4bf1-a92b-6a8c83034f46 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:15.602392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d06d4-d1fd-4bf1-a92b-6a8c83034f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:15.602666 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d06d4-d1fd-4bf1-a92b-6a8c83034f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:15.603394 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d06d4-d1fd-4bf1-a92b-6a8c83034f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:15.605731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d06d4-d1fd-4bf1-a92b-6a8c83034f46 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:15.616120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 816/1637] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:15.620541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2b5de-88d4-4ae4-8f7a-5b49b1b6bccc admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:15.650506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2b5de-88d4-4ae4-8f7a-5b49b1b6bccc admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:15.658533 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b60e3c-ae71-4fb2-86cf-d97fadec8f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:15.659305 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b60e3c-ae71-4fb2-86cf-d97fadec8f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:15.659592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b60e3c-ae71-4fb2-86cf-d97fadec8f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:15.659977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b60e3c-ae71-4fb2-86cf-d97fadec8f22 admin admin] PATH_INFO: `/v3/projects/4d8645f9acf14d038ffacccc378eaf9f/users/538b2f4067b541d5a16a0b67caf0e73d/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:15.664602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b60e3c-ae71-4fb2-86cf-d97fadec8f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d8645f9acf14d038ffacccc378eaf9f, user_id=538b2f4067b541d5a16a0b67caf0e73d, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:15.670281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b60e3c-ae71-4fb2-86cf-d97fadec8f22 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:15.706167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 822/1638] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:15 2022] PUT /identity/v3/projects/4d8645f9acf14d038ffacccc378eaf9f/users/538b2f4067b541d5a16a0b67caf0e73d/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:16.812114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17333c2-54e2-4b70-9025-abe3410722d0 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:16.812628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17333c2-54e2-4b70-9025-abe3410722d0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:16.812698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17333c2-54e2-4b70-9025-abe3410722d0 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:16.814026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 817/1639] 10.222.0.73 () {54 vars in 962 bytes} [Thu Aug 4 21:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:40:16.820558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ef8a2-6511-4cb0-bf1b-66e62cda5661 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:16.821080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ef8a2-6511-4cb0-bf1b-66e62cda5661 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:16.821080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ef8a2-6511-4cb0-bf1b-66e62cda5661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:16.848796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-436ef8a2-6511-4cb0-bf1b-66e62cda5661 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:16.850889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-436ef8a2-6511-4cb0-bf1b-66e62cda5661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:16.942256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 823/1640] 10.222.0.73 () {58 vars in 1059 bytes} [Thu Aug 4 21:40:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:16.947772 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a776022-5aab-4299-910b-cc3cfb7deda8 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:16.948068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a776022-5aab-4299-910b-cc3cfb7deda8 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:16.948864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a776022-5aab-4299-910b-cc3cfb7deda8 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:16.950653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 818/1641] 10.222.0.73 () {54 vars in 912 bytes} [Thu Aug 4 21:40:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:40:16.955867 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46411382-5d53-4ead-ab97-3265e007ed67 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:16.956327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46411382-5d53-4ead-ab97-3265e007ed67 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:16.956733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46411382-5d53-4ead-ab97-3265e007ed67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:16.965789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a776022-5aab-4299-910b-cc3cfb7deda8 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:17.010064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-46411382-5d53-4ead-ab97-3265e007ed67 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:17.014556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-46411382-5d53-4ead-ab97-3265e007ed67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:17.034973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 824/1642] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:40:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:17.061792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46411382-5d53-4ead-ab97-3265e007ed67 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:17.147024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a776022-5aab-4299-910b-cc3cfb7deda8 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:17.175020 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:17.179695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:17.179695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:17.179695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] PATH_INFO: `/v3/users/1d016173354445d699d0007610ee0144` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:17.179695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d016173354445d699d0007610ee0144)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:17.181431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:17.215188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46411382-5d53-4ead-ab97-3265e007ed67 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:17.225012 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3af34-0e4a-4da9-b71e-4e4ad3c04547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:17.225943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3af34-0e4a-4da9-b71e-4e4ad3c04547 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:17.226886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3af34-0e4a-4da9-b71e-4e4ad3c04547 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:17.226886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3af34-0e4a-4da9-b71e-4e4ad3c04547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:17.242180 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3af34-0e4a-4da9-b71e-4e4ad3c04547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:17.244575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3af34-0e4a-4da9-b71e-4e4ad3c04547 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:17.287853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 825/1643] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:40:17.451199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d016173354445d699d0007610ee0144'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:17.545082 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d016173354445d699d0007610ee0144'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:17.579840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d016173354445d699d0007610ee0144'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:17.580644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d016173354445d699d0007610ee0144'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:17.601971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d016173354445d699d0007610ee0144'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:17.614803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 819/1644] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:16 2022] DELETE /identity/v3/users/1d016173354445d699d0007610ee0144 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:17.620001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc9596-b386-44a7-b881-bb5d0b1f64da None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:17.620579 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc9596-b386-44a7-b881-bb5d0b1f64da None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:17.621194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc9596-b386-44a7-b881-bb5d0b1f64da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:17.657001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-cdcc9596-b386-44a7-b881-bb5d0b1f64da None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:17.658528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-cdcc9596-b386-44a7-b881-bb5d0b1f64da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:17.666614 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:17.776407 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 826/1645] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:17.796348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:17.797961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:17.836022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:17.878312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7fc51-9585-4a46-bc62-69adfef9b0f0 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:17.884511 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d68e8-4e46-4b0b-9b2a-f886e9629383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:17.885529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d68e8-4e46-4b0b-9b2a-f886e9629383 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:17.887281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d68e8-4e46-4b0b-9b2a-f886e9629383 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:17.887644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d68e8-4e46-4b0b-9b2a-f886e9629383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:17.899475 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d68e8-4e46-4b0b-9b2a-f886e9629383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:17.902288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d68e8-4e46-4b0b-9b2a-f886e9629383 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:17.921179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 820/1646] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:18.905857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01932a78-8fca-4c50-b2b7-a9e0170f9ade None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:18.906671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01932a78-8fca-4c50-b2b7-a9e0170f9ade None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:18.906671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01932a78-8fca-4c50-b2b7-a9e0170f9ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:18.956375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-01932a78-8fca-4c50-b2b7-a9e0170f9ade None None] MFA Rules not processed for user `538b2f4067b541d5a16a0b67caf0e73d`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:18.957980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-01932a78-8fca-4c50-b2b7-a9e0170f9ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:19.093056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 827/1647] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:18 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:19.466480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d68e8-4e46-4b0b-9b2a-f886e9629383 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:19.468704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-622d68e8-4e46-4b0b-9b2a-f886e9629383 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:19.490066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d68e8-4e46-4b0b-9b2a-f886e9629383 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:19.497392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:19.498453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:19.499038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:19.500408 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] PATH_INFO: `/v3/projects/138cb60729014ec599089a01fba5553f` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:19.502107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=138cb60729014ec599089a01fba5553f)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:19.503363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: WARNING py.warnings [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:40:19.503363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: warnings.warn(msg) Aug 04 21:40:19.503363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: Aug 04 21:40:19.504971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:19.751743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '138cb60729014ec599089a01fba5553f', 'request_id': 'req-5f540957-f34e-421d-95e9-7237a4876903'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:19.851125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '138cb60729014ec599089a01fba5553f', 'request_id': 'req-5f540957-f34e-421d-95e9-7237a4876903'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:19.869650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '138cb60729014ec599089a01fba5553f', 'request_id': 'req-5f540957-f34e-421d-95e9-7237a4876903'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:19.931323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] The token cache is being invalidate because project 138cb60729014ec599089a01fba5553f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:19.931683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:19.933682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:19.938273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01932a78-8fca-4c50-b2b7-a9e0170f9ade None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:19.939065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 821/1648] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:19 2022] DELETE /identity/v3/projects/138cb60729014ec599089a01fba5553f => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:19.958323 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01932a78-8fca-4c50-b2b7-a9e0170f9ade None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:20.124922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:20.128915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:20.128915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:20.128915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] PATH_INFO: `/v3/users/6193dd552d6e4194b7e9f41103c865b6` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:20.133559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6193dd552d6e4194b7e9f41103c865b6)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:20.134812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: WARNING py.warnings [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 21:40:20.134812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: warnings.warn(msg) Aug 04 21:40:20.134812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: Aug 04 21:40:20.136697 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:20.432556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:20.460959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6193dd552d6e4194b7e9f41103c865b6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:20.617573 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6193dd552d6e4194b7e9f41103c865b6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:20.617915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6193dd552d6e4194b7e9f41103c865b6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:20.690978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6193dd552d6e4194b7e9f41103c865b6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:20.742969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f1ed0131-a01b-4caa-bd90-71df244e283e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6193dd552d6e4194b7e9f41103c865b6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:20.778731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 828/1649] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:19 2022] DELETE /identity/v3/users/6193dd552d6e4194b7e9f41103c865b6 => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:22.384263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:22.386160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:22.412442 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac73af-9118-4a9c-9906-d4679b71e5da None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:22.424822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac73af-9118-4a9c-9906-d4679b71e5da None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:22.424822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac73af-9118-4a9c-9906-d4679b71e5da None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:22.424822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 829/1650] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:40:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:40:22.425281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f540957-f34e-421d-95e9-7237a4876903 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:22.467282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7d1b4-c9cd-41af-bd28-083aa29e7727 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:22.467764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7d1b4-c9cd-41af-bd28-083aa29e7727 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:22.468308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7d1b4-c9cd-41af-bd28-083aa29e7727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:22.522632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0df7d1b4-c9cd-41af-bd28-083aa29e7727 None None] MFA Rules not processed for user `fab36b186fa44ee18bf72228fc7ba758`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:22.530256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0df7d1b4-c9cd-41af-bd28-083aa29e7727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:22.603760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:22.607835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:22.608061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:22.610552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] PATH_INFO: `/v3/projects/e53decc6afec43cd8e1e5f74d6f0f858` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:22.613190 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e53decc6afec43cd8e1e5f74d6f0f858)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:22.618676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:22.728541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 830/1651] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:40:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:22.796724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7d1b4-c9cd-41af-bd28-083aa29e7727 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:22.799516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0df7d1b4-c9cd-41af-bd28-083aa29e7727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:22.843170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7d1b4-c9cd-41af-bd28-083aa29e7727 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:22.862701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e53decc6afec43cd8e1e5f74d6f0f858', 'request_id': 'req-24434548-07e7-4d23-acd6-574240f52812'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:22.867534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:22.940380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e53decc6afec43cd8e1e5f74d6f0f858', 'request_id': 'req-24434548-07e7-4d23-acd6-574240f52812'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:22.949143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e53decc6afec43cd8e1e5f74d6f0f858', 'request_id': 'req-24434548-07e7-4d23-acd6-574240f52812'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:22.983259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] The token cache is being invalidate because project e53decc6afec43cd8e1e5f74d6f0f858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:22.983556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:22.985746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24434548-07e7-4d23-acd6-574240f52812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:22.993373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 822/1652] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:22 2022] DELETE /identity/v3/projects/e53decc6afec43cd8e1e5f74d6f0f858 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:22.999932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba80b0-6e21-4c25-a9a8-91b90dfa0813 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:23.001500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba80b0-6e21-4c25-a9a8-91b90dfa0813 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:23.002349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba80b0-6e21-4c25-a9a8-91b90dfa0813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:23.003604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27daf6-2d89-47a0-b35e-43086b128329 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:23.013734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27daf6-2d89-47a0-b35e-43086b128329 service placement] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:23.013734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27daf6-2d89-47a0-b35e-43086b128329 service placement] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:23.013734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27daf6-2d89-47a0-b35e-43086b128329 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:23.015755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9b27daf6-2d89-47a0-b35e-43086b128329 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:23.038336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27daf6-2d89-47a0-b35e-43086b128329 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:23.041659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27daf6-2d89-47a0-b35e-43086b128329 service placement] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:23.048683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-5dba80b0-6e21-4c25-a9a8-91b90dfa0813 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:23.050112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5dba80b0-6e21-4c25-a9a8-91b90dfa0813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:23.070111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 831/1653] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:40:23.200582 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 823/1654] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:23.207710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1903e5f-5b52-4741-91f9-0dc6a285027e None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:23.208010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1903e5f-5b52-4741-91f9-0dc6a285027e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:23.208289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1903e5f-5b52-4741-91f9-0dc6a285027e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:23.248058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-e1903e5f-5b52-4741-91f9-0dc6a285027e None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:23.249325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e1903e5f-5b52-4741-91f9-0dc6a285027e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:23.274483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 832/1655] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:23.274821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba80b0-6e21-4c25-a9a8-91b90dfa0813 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:23.296170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba80b0-6e21-4c25-a9a8-91b90dfa0813 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:23.312524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326a817-6cd7-4910-8220-5ad594cd0c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:23.314031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326a817-6cd7-4910-8220-5ad594cd0c30 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:23.314772 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326a817-6cd7-4910-8220-5ad594cd0c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:23.315225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326a817-6cd7-4910-8220-5ad594cd0c30 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:23.316657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326a817-6cd7-4910-8220-5ad594cd0c30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:23.317401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326a817-6cd7-4910-8220-5ad594cd0c30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:23.321607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326a817-6cd7-4910-8220-5ad594cd0c30 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:23.345036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 824/1656] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:23.349100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1903e5f-5b52-4741-91f9-0dc6a285027e None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:23.363199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1903e5f-5b52-4741-91f9-0dc6a285027e None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:23.371724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00bf67-bd62-4c2b-ba5d-3236d9dbdb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:23.372656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00bf67-bd62-4c2b-ba5d-3236d9dbdb07 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:23.373156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00bf67-bd62-4c2b-ba5d-3236d9dbdb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:23.373643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00bf67-bd62-4c2b-ba5d-3236d9dbdb07 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:23.375826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00bf67-bd62-4c2b-ba5d-3236d9dbdb07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:23.378379 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00bf67-bd62-4c2b-ba5d-3236d9dbdb07 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:23.525331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 833/1657] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:23 2022] POST /identity/v3/projects => generated 369 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:23.530721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326a817-6cd7-4910-8220-5ad594cd0c30 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:23.553864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326a817-6cd7-4910-8220-5ad594cd0c30 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:23.646607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcc137-9168-47cd-a675-f98afa4c5873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:23.647241 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcc137-9168-47cd-a675-f98afa4c5873 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:23.647679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcc137-9168-47cd-a675-f98afa4c5873 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:23.648135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcc137-9168-47cd-a675-f98afa4c5873 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:23.653495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bcc137-9168-47cd-a675-f98afa4c5873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:23.655656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bcc137-9168-47cd-a675-f98afa4c5873 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:23.765120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 825/1658] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:23 2022] POST /identity/v3/users => generated 285 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:23.769729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00bf67-bd62-4c2b-ba5d-3236d9dbdb07 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:23.795699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00bf67-bd62-4c2b-ba5d-3236d9dbdb07 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:23.808559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1a7ac-dd1e-48c4-bd36-e22d45ae210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:23.809632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1a7ac-dd1e-48c4-bd36-e22d45ae210b admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:23.810500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1a7ac-dd1e-48c4-bd36-e22d45ae210b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:23.810940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1a7ac-dd1e-48c4-bd36-e22d45ae210b admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:23.814181 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1a7ac-dd1e-48c4-bd36-e22d45ae210b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:23.817989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1a7ac-dd1e-48c4-bd36-e22d45ae210b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:23.856421 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcc137-9168-47cd-a675-f98afa4c5873 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:23.868208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 834/1659] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:23.887039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcc137-9168-47cd-a675-f98afa4c5873 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:23.905326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fa8f3-ff90-40f8-b009-86918d66af4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:23.906271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fa8f3-ff90-40f8-b009-86918d66af4e admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:23.906271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fa8f3-ff90-40f8-b009-86918d66af4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:23.906534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fa8f3-ff90-40f8-b009-86918d66af4e admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:23.907245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fa8f3-ff90-40f8-b009-86918d66af4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:23.913696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fa8f3-ff90-40f8-b009-86918d66af4e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:23.962060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 826/1660] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:23.966635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1a7ac-dd1e-48c4-bd36-e22d45ae210b admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:23.972435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:23.982098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1a7ac-dd1e-48c4-bd36-e22d45ae210b admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:23.990444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0634b55-d20d-4346-bf14-a94c5b642a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:23.991686 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0634b55-d20d-4346-bf14-a94c5b642a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:23.992202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0634b55-d20d-4346-bf14-a94c5b642a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:23.992829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0634b55-d20d-4346-bf14-a94c5b642a54 admin admin] PATH_INFO: `/v3/projects/38baafa2203044b9bec92dc99e18d36c/users/4263fdd2cf4e4a46a38adba3c41d1dae/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:24.019056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0634b55-d20d-4346-bf14-a94c5b642a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38baafa2203044b9bec92dc99e18d36c, user_id=4263fdd2cf4e4a46a38adba3c41d1dae, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:24.023642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0634b55-d20d-4346-bf14-a94c5b642a54 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:24.090903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 835/1661] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:23 2022] PUT /identity/v3/projects/38baafa2203044b9bec92dc99e18d36c/users/4263fdd2cf4e4a46a38adba3c41d1dae/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:24.101291 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8957983-50a0-4caa-9236-0cdbb8113cea None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:24.107213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8957983-50a0-4caa-9236-0cdbb8113cea None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:24.108523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8957983-50a0-4caa-9236-0cdbb8113cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:24.259031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f8957983-50a0-4caa-9236-0cdbb8113cea None None] MFA Rules not processed for user `4263fdd2cf4e4a46a38adba3c41d1dae`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:24.262592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f8957983-50a0-4caa-9236-0cdbb8113cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:24.342103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:24.441385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0634b55-d20d-4346-bf14-a94c5b642a54 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:24.441912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 827/1662] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:24.607731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0634b55-d20d-4346-bf14-a94c5b642a54 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:24.615689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6a621-8804-4791-96d0-2e83ae5400d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:24.617548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a621-8804-4791-96d0-2e83ae5400d8 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:24.618042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a621-8804-4791-96d0-2e83ae5400d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:24.618532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a621-8804-4791-96d0-2e83ae5400d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:24.619385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6a621-8804-4791-96d0-2e83ae5400d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:24.622917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6a621-8804-4791-96d0-2e83ae5400d8 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:24.726093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 836/1663] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:24 2022] POST /identity/v3/projects => generated 369 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:24.729729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8957983-50a0-4caa-9236-0cdbb8113cea None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:24.754899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8957983-50a0-4caa-9236-0cdbb8113cea None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:24.893533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497efe-1363-461d-a53b-4f8a4e340e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:24.893980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497efe-1363-461d-a53b-4f8a4e340e28 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:24.894275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497efe-1363-461d-a53b-4f8a4e340e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:24.894551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497efe-1363-461d-a53b-4f8a4e340e28 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:24.895236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4497efe-1363-461d-a53b-4f8a4e340e28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:24.903905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4497efe-1363-461d-a53b-4f8a4e340e28 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:24.930774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6a621-8804-4791-96d0-2e83ae5400d8 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:24.933215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-41a6a621-8804-4791-96d0-2e83ae5400d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:24.956177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6a621-8804-4791-96d0-2e83ae5400d8 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:24.966879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:24.967569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:24.967861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:24.968145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] PATH_INFO: `/v3/users/7c3d7bae0f004eac9bfe3770e2425091` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:24.973541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c3d7bae0f004eac9bfe3770e2425091)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:24.975654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:25.024852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 828/1664] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:24 2022] POST /identity/v3/users => generated 291 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:25.026349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497efe-1363-461d-a53b-4f8a4e340e28 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:25.052614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497efe-1363-461d-a53b-4f8a4e340e28 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:25.064657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35ecd5-e630-40af-81e6-0b8efcc02f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:25.065299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35ecd5-e630-40af-81e6-0b8efcc02f8c admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:25.065641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35ecd5-e630-40af-81e6-0b8efcc02f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:25.065923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35ecd5-e630-40af-81e6-0b8efcc02f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:25.066656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35ecd5-e630-40af-81e6-0b8efcc02f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:25.071532 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35ecd5-e630-40af-81e6-0b8efcc02f8c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:25.101766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 829/1665] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:25.102705 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35ecd5-e630-40af-81e6-0b8efcc02f8c admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:25.126137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35ecd5-e630-40af-81e6-0b8efcc02f8c admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:25.140578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030b852-b012-4954-9bbd-859ebf76483a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:25.141286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030b852-b012-4954-9bbd-859ebf76483a admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:25.141572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030b852-b012-4954-9bbd-859ebf76483a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:25.141848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030b852-b012-4954-9bbd-859ebf76483a admin admin] PATH_INFO: `/v3/projects/61cc92d7a4ad483a9970cf5b502df467/users/1e1fe30eafce48278f3d267ecd08fe0a/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:25.177168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c3d7bae0f004eac9bfe3770e2425091'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:25.180821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030b852-b012-4954-9bbd-859ebf76483a admin admin] RBAC: Authorizing `identity:create_grant(project_id=61cc92d7a4ad483a9970cf5b502df467, user_id=1e1fe30eafce48278f3d267ecd08fe0a, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:25.184320 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030b852-b012-4954-9bbd-859ebf76483a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:25.256579 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c3d7bae0f004eac9bfe3770e2425091'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:25.257246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c3d7bae0f004eac9bfe3770e2425091'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:25.287766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c3d7bae0f004eac9bfe3770e2425091'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:25.294650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 830/1666] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:25 2022] PUT /identity/v3/projects/61cc92d7a4ad483a9970cf5b502df467/users/1e1fe30eafce48278f3d267ecd08fe0a/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:25.299147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030b852-b012-4954-9bbd-859ebf76483a admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:25.310232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c3d7bae0f004eac9bfe3770e2425091'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:25.329424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 837/1667] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:24 2022] DELETE /identity/v3/users/7c3d7bae0f004eac9bfe3770e2425091 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:25.404674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030b852-b012-4954-9bbd-859ebf76483a admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:25.547563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedd472-2a9c-47e5-8729-e0fb5ecb9a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:25.549690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedd472-2a9c-47e5-8729-e0fb5ecb9a5c admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:25.550606 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedd472-2a9c-47e5-8729-e0fb5ecb9a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:25.551026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedd472-2a9c-47e5-8729-e0fb5ecb9a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:25.553424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedd472-2a9c-47e5-8729-e0fb5ecb9a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:25.556123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedd472-2a9c-47e5-8729-e0fb5ecb9a5c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:25.579055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 831/1668] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:25.583319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:25.585861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:25.611770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d979c-1f32-4f78-9291-899a4f57da86 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:25.621447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67604c-2ffb-4b18-b466-10080bb4d3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:25.622425 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67604c-2ffb-4b18-b466-10080bb4d3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:25.623740 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67604c-2ffb-4b18-b466-10080bb4d3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:25.624177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67604c-2ffb-4b18-b466-10080bb4d3d6 admin admin] PATH_INFO: `/v3/projects/61cc92d7a4ad483a9970cf5b502df467/users/1e1fe30eafce48278f3d267ecd08fe0a/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:25.626703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67604c-2ffb-4b18-b466-10080bb4d3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61cc92d7a4ad483a9970cf5b502df467, user_id=1e1fe30eafce48278f3d267ecd08fe0a, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:25.632125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67604c-2ffb-4b18-b466-10080bb4d3d6 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:25.672547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 838/1669] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:25 2022] PUT /identity/v3/projects/61cc92d7a4ad483a9970cf5b502df467/users/1e1fe30eafce48278f3d267ecd08fe0a/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:25.674996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde002b-c65b-4c03-a25d-39a2b1400913 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:25.675286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde002b-c65b-4c03-a25d-39a2b1400913 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:25.675565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde002b-c65b-4c03-a25d-39a2b1400913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:25.727247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-2fde002b-c65b-4c03-a25d-39a2b1400913 None None] MFA Rules not processed for user `1e1fe30eafce48278f3d267ecd08fe0a`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:25.730918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2fde002b-c65b-4c03-a25d-39a2b1400913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:25.848507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 832/1670] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:25.856869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67604c-2ffb-4b18-b466-10080bb4d3d6 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:25.858774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3e67604c-2ffb-4b18-b466-10080bb4d3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:25.921394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:26.028430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67604c-2ffb-4b18-b466-10080bb4d3d6 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:26.045913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb4dba-1105-4b59-9d47-f19818daba9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:26.046707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb4dba-1105-4b59-9d47-f19818daba9a service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:26.047490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb4dba-1105-4b59-9d47-f19818daba9a service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:26.048249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb4dba-1105-4b59-9d47-f19818daba9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:26.072746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb4dba-1105-4b59-9d47-f19818daba9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:26.074981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb4dba-1105-4b59-9d47-f19818daba9a service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:26.090718 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 839/1671] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:26.098334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:26.309786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde002b-c65b-4c03-a25d-39a2b1400913 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:26.312147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2fde002b-c65b-4c03-a25d-39a2b1400913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:26.346048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde002b-c65b-4c03-a25d-39a2b1400913 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:26.361042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b346413-8d47-47b6-98c8-d800a0b8f065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:26.364455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b346413-8d47-47b6-98c8-d800a0b8f065 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:26.365797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b346413-8d47-47b6-98c8-d800a0b8f065 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:26.366711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b346413-8d47-47b6-98c8-d800a0b8f065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:26.375707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7b346413-8d47-47b6-98c8-d800a0b8f065 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:26.605795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b346413-8d47-47b6-98c8-d800a0b8f065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:26.612624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b346413-8d47-47b6-98c8-d800a0b8f065 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:26.628682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 833/1672] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:26.643334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad873c-5cf5-4f57-a3ee-3c63b4d5b87c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:26.648434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad873c-5cf5-4f57-a3ee-3c63b4d5b87c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:26.648805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad873c-5cf5-4f57-a3ee-3c63b4d5b87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:26.730501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d7ad873c-5cf5-4f57-a3ee-3c63b4d5b87c None None] MFA Rules not processed for user `a23fcabad2354e7ab51a22b412bf11e6`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:26.738999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d7ad873c-5cf5-4f57-a3ee-3c63b4d5b87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:26.808463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b346413-8d47-47b6-98c8-d800a0b8f065 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:26.810064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7b346413-8d47-47b6-98c8-d800a0b8f065 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:26.960605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 840/1673] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:26.976766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad873c-5cf5-4f57-a3ee-3c63b4d5b87c None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:26.978460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d7ad873c-5cf5-4f57-a3ee-3c63b4d5b87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:27.017041 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b346413-8d47-47b6-98c8-d800a0b8f065 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:27.033022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:27.034038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:27.037080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:27.041034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] PATH_INFO: `/v3/projects/0cdb0d7348374f9aba8720777b847b23` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:27.044796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cdb0d7348374f9aba8720777b847b23)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:27.054603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:27.165735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad873c-5cf5-4f57-a3ee-3c63b4d5b87c None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:27.178752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e7684-ffba-4406-aa0b-7108d943eacb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:27.179413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e7684-ffba-4406-aa0b-7108d943eacb service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:27.179697 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e7684-ffba-4406-aa0b-7108d943eacb service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:27.179974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e7684-ffba-4406-aa0b-7108d943eacb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:27.210878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e7684-ffba-4406-aa0b-7108d943eacb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:27.224487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e7684-ffba-4406-aa0b-7108d943eacb service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:27.225383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cdb0d7348374f9aba8720777b847b23', 'request_id': 'req-e1f71ea4-0951-4135-868a-685b4906b0b9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:27.255567 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:27.291456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 841/1674] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:40:27.341107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cdb0d7348374f9aba8720777b847b23', 'request_id': 'req-e1f71ea4-0951-4135-868a-685b4906b0b9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:27.354755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cdb0d7348374f9aba8720777b847b23', 'request_id': 'req-e1f71ea4-0951-4135-868a-685b4906b0b9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:27.394715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] The token cache is being invalidate because project 0cdb0d7348374f9aba8720777b847b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:27.395894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:27.396788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-e1f71ea4-0951-4135-868a-685b4906b0b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:27.402095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 834/1675] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:26 2022] DELETE /identity/v3/projects/0cdb0d7348374f9aba8720777b847b23 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:27.405906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e7684-ffba-4406-aa0b-7108d943eacb service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:27.437595 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e7684-ffba-4406-aa0b-7108d943eacb service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:27.551672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:27.583938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b133df6-4e97-4f5c-94e6-0f9a8b63f7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:27.584792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b133df6-4e97-4f5c-94e6-0f9a8b63f7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:27.585195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b133df6-4e97-4f5c-94e6-0f9a8b63f7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:27.585588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b133df6-4e97-4f5c-94e6-0f9a8b63f7f0 admin admin] PATH_INFO: `/v3/users/538b2f4067b541d5a16a0b67caf0e73d` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:27.592763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b133df6-4e97-4f5c-94e6-0f9a8b63f7f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=538b2f4067b541d5a16a0b67caf0e73d)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:27.595839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b133df6-4e97-4f5c-94e6-0f9a8b63f7f0 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:27.959374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7b133df6-4e97-4f5c-94e6-0f9a8b63f7f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '538b2f4067b541d5a16a0b67caf0e73d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:28.081020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7b133df6-4e97-4f5c-94e6-0f9a8b63f7f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '538b2f4067b541d5a16a0b67caf0e73d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:28.081677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7b133df6-4e97-4f5c-94e6-0f9a8b63f7f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '538b2f4067b541d5a16a0b67caf0e73d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:28.107160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7b133df6-4e97-4f5c-94e6-0f9a8b63f7f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '538b2f4067b541d5a16a0b67caf0e73d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:28.129082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7b133df6-4e97-4f5c-94e6-0f9a8b63f7f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '538b2f4067b541d5a16a0b67caf0e73d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:28.144479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 842/1676] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:27 2022] DELETE /identity/v3/users/538b2f4067b541d5a16a0b67caf0e73d => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:30.899017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a7759-d423-4d7e-9e20-8d0490ace83c None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:30.899017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a7759-d423-4d7e-9e20-8d0490ace83c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:30.899600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a7759-d423-4d7e-9e20-8d0490ace83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:30.919909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee57030-c408-4e10-a220-73374fc140c7 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:30.920407 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee57030-c408-4e10-a220-73374fc140c7 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:30.920554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee57030-c408-4e10-a220-73374fc140c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:31.056747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-6c0a7759-d423-4d7e-9e20-8d0490ace83c None None] MFA Rules not processed for user `95fb2edf82644e7f8ce461fcbb52007b`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:31.065663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6c0a7759-d423-4d7e-9e20-8d0490ace83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:31.095622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0ee57030-c408-4e10-a220-73374fc140c7 None None] MFA Rules not processed for user `d9bc42bec393430b9b3e9399a8c4f95e`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:31.097226 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0ee57030-c408-4e10-a220-73374fc140c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:31.263678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 835/1677] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:31.264803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a7759-d423-4d7e-9e20-8d0490ace83c None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:31.266254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6c0a7759-d423-4d7e-9e20-8d0490ace83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:31.320091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 843/1678] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:31.321292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee57030-c408-4e10-a220-73374fc140c7 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:31.324534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0ee57030-c408-4e10-a220-73374fc140c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:31.400418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a7759-d423-4d7e-9e20-8d0490ace83c None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:31.408586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:31.409852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:31.410906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:31.411193 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] PATH_INFO: `/v3/projects/4d8645f9acf14d038ffacccc378eaf9f` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:31.412939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d8645f9acf14d038ffacccc378eaf9f)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:31.415423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:31.422276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:31.452861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee57030-c408-4e10-a220-73374fc140c7 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:31.453279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:31.465683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:31.467159 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:31.471891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:31.471891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] PATH_INFO: `/v3/users/cd348c0520e246d5814e035349bf7b90` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:31.473588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd348c0520e246d5814e035349bf7b90)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:31.475826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:31.577632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d8645f9acf14d038ffacccc378eaf9f', 'request_id': 'req-71c3f889-5cc9-4cf0-9322-c7f016331aeb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:31.663692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d8645f9acf14d038ffacccc378eaf9f', 'request_id': 'req-71c3f889-5cc9-4cf0-9322-c7f016331aeb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:31.671548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d8645f9acf14d038ffacccc378eaf9f', 'request_id': 'req-71c3f889-5cc9-4cf0-9322-c7f016331aeb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:31.697165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] The token cache is being invalidate because project 4d8645f9acf14d038ffacccc378eaf9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:31.697711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:31.698674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-71c3f889-5cc9-4cf0-9322-c7f016331aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:31.704128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 836/1679] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:31 2022] DELETE /identity/v3/projects/4d8645f9acf14d038ffacccc378eaf9f => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:31.705414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203cbdf-9864-41ff-ae46-54dfc06db69a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:31.705587 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203cbdf-9864-41ff-ae46-54dfc06db69a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:31.705856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203cbdf-9864-41ff-ae46-54dfc06db69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:31.742912 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd348c0520e246d5814e035349bf7b90'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:31.760870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-0203cbdf-9864-41ff-ae46-54dfc06db69a None None] MFA Rules not processed for user `d681a6cc3e5247c9bb2537a6cacb7dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:31.762663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-0203cbdf-9864-41ff-ae46-54dfc06db69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:31.822166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd348c0520e246d5814e035349bf7b90'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:31.822913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd348c0520e246d5814e035349bf7b90'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:31.852693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd348c0520e246d5814e035349bf7b90'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:31.879070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd348c0520e246d5814e035349bf7b90'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:31.895311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 844/1680] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:31 2022] DELETE /identity/v3/users/cd348c0520e246d5814e035349bf7b90 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:31.898524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:31.899535 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:31.902404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 837/1681] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:31.903940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90e5fa-f191-4416-967b-374a93d5e7cd None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:31.904239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90e5fa-f191-4416-967b-374a93d5e7cd None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:31.904520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90e5fa-f191-4416-967b-374a93d5e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:31.940105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ae90e5fa-f191-4416-967b-374a93d5e7cd None None] MFA Rules not processed for user `d9bc42bec393430b9b3e9399a8c4f95e`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:31.941448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ae90e5fa-f191-4416-967b-374a93d5e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:32.099291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a519277-3bea-4614-8c85-59d6462cd723 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:32.112706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7366de3-78c6-42d7-8fe4-f7a4b5e1a34b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:32.119842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 838/1682] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:32.123181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6cd41-c855-42f4-94c1-cdd07f724e40 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:32.123181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6cd41-c855-42f4-94c1-cdd07f724e40 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:32.123391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366de3-78c6-42d7-8fe4-f7a4b5e1a34b service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:32.123854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366de3-78c6-42d7-8fe4-f7a4b5e1a34b service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:32.124447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366de3-78c6-42d7-8fe4-f7a4b5e1a34b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:32.125648 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6cd41-c855-42f4-94c1-cdd07f724e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:32.142145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a7366de3-78c6-42d7-8fe4-f7a4b5e1a34b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:32.189914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-2af6cd41-c855-42f4-94c1-cdd07f724e40 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:32.192427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2af6cd41-c855-42f4-94c1-cdd07f724e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:32.324690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7366de3-78c6-42d7-8fe4-f7a4b5e1a34b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:32.328200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7366de3-78c6-42d7-8fe4-f7a4b5e1a34b service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:32.386546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 845/1683] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:32.394969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9eabb3-c1a5-4c34-a8a0-ea698ff88b4e None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:32.396151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9eabb3-c1a5-4c34-a8a0-ea698ff88b4e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:32.396622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9eabb3-c1a5-4c34-a8a0-ea698ff88b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:32.418601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 839/1684] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:32.425689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b664e7-60fb-47f6-b3ef-956717b62b4a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:32.427107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b664e7-60fb-47f6-b3ef-956717b62b4a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:32.431594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b664e7-60fb-47f6-b3ef-956717b62b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:32.465981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-4a9eabb3-c1a5-4c34-a8a0-ea698ff88b4e None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:32.467703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4a9eabb3-c1a5-4c34-a8a0-ea698ff88b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:32.483392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-63b664e7-60fb-47f6-b3ef-956717b62b4a None None] MFA Rules not processed for user `d681a6cc3e5247c9bb2537a6cacb7dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:32.489248 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-63b664e7-60fb-47f6-b3ef-956717b62b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:32.496422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 846/1685] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:32.498733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9eabb3-c1a5-4c34-a8a0-ea698ff88b4e None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:32.500484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4a9eabb3-c1a5-4c34-a8a0-ea698ff88b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:32.554879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9eabb3-c1a5-4c34-a8a0-ea698ff88b4e None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:32.562583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd5574-e868-4bba-9238-54a039504533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:32.563071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd5574-e868-4bba-9238-54a039504533 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:32.563524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd5574-e868-4bba-9238-54a039504533 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:32.563803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd5574-e868-4bba-9238-54a039504533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:32.566182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-abcd5574-e868-4bba-9238-54a039504533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:32.605949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 840/1686] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:32.608647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd5574-e868-4bba-9238-54a039504533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:32.610598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd5574-e868-4bba-9238-54a039504533 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:32.623891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 847/1687] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:32.624067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e1b66-81ce-4424-acc7-a3f6c15bf0dd None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:32.624067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e1b66-81ce-4424-acc7-a3f6c15bf0dd None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:32.625136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd5574-e868-4bba-9238-54a039504533 service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:32.627443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e1b66-81ce-4424-acc7-a3f6c15bf0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:32.627562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-abcd5574-e868-4bba-9238-54a039504533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:32.703527 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-290e1b66-81ce-4424-acc7-a3f6c15bf0dd None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:32.705774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-290e1b66-81ce-4424-acc7-a3f6c15bf0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:32.726475 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 841/1688] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:32.730455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e1b66-81ce-4424-acc7-a3f6c15bf0dd None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:32.761720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e1b66-81ce-4424-acc7-a3f6c15bf0dd None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:32.774654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa6f95-a8fc-48d9-a2ee-53d20389602c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:32.775378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa6f95-a8fc-48d9-a2ee-53d20389602c service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:32.775679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa6f95-a8fc-48d9-a2ee-53d20389602c service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:32.776875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa6f95-a8fc-48d9-a2ee-53d20389602c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:32.813948 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa6f95-a8fc-48d9-a2ee-53d20389602c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:32.816556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa6f95-a8fc-48d9-a2ee-53d20389602c service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:32.834814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 842/1689] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:32.836656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa6f95-a8fc-48d9-a2ee-53d20389602c service nova] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:32.853011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa6f95-a8fc-48d9-a2ee-53d20389602c service nova] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:32.863198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd5574-e868-4bba-9238-54a039504533 service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:32.870343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98223ebe-2352-4a3a-ad6e-d2dc8224ee6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:32.870343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98223ebe-2352-4a3a-ad6e-d2dc8224ee6c service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:32.870343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98223ebe-2352-4a3a-ad6e-d2dc8224ee6c service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:32.870343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98223ebe-2352-4a3a-ad6e-d2dc8224ee6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:32.885189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b9676-e80a-4d2e-9ecc-5ed7867adfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:32.888668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b9676-e80a-4d2e-9ecc-5ed7867adfb9 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:32.889142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b9676-e80a-4d2e-9ecc-5ed7867adfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:32.890407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b9676-e80a-4d2e-9ecc-5ed7867adfb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:32.892563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98223ebe-2352-4a3a-ad6e-d2dc8224ee6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:32.893317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b9676-e80a-4d2e-9ecc-5ed7867adfb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:32.894679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b9676-e80a-4d2e-9ecc-5ed7867adfb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:32.895490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98223ebe-2352-4a3a-ad6e-d2dc8224ee6c service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:32.899128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b9676-e80a-4d2e-9ecc-5ed7867adfb9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:32.930092 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 848/1690] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:32.935302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 843/1691] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:32.939602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98223ebe-2352-4a3a-ad6e-d2dc8224ee6c service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:32.955735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98223ebe-2352-4a3a-ad6e-d2dc8224ee6c service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:32.980615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0df63b-5d13-40fd-9c71-62151c612321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:32.982430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0df63b-5d13-40fd-9c71-62151c612321 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:32.982863 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0df63b-5d13-40fd-9c71-62151c612321 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:32.985534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0df63b-5d13-40fd-9c71-62151c612321 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:32.986574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0df63b-5d13-40fd-9c71-62151c612321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:32.990601 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0df63b-5d13-40fd-9c71-62151c612321 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:33.161229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 849/1692] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:32 2022] POST /identity/v3/projects => generated 379 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:33.161915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b9676-e80a-4d2e-9ecc-5ed7867adfb9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:33.199664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b9676-e80a-4d2e-9ecc-5ed7867adfb9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:33.378639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe4791-58f0-41ec-8514-8a391de3a07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:33.382397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe4791-58f0-41ec-8514-8a391de3a07f admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:33.386080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe4791-58f0-41ec-8514-8a391de3a07f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:33.386080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe4791-58f0-41ec-8514-8a391de3a07f admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:33.386080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe4791-58f0-41ec-8514-8a391de3a07f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:33.388098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe4791-58f0-41ec-8514-8a391de3a07f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:33.509467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 844/1693] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:33 2022] POST /identity/v3/users => generated 290 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:33.511133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0df63b-5d13-40fd-9c71-62151c612321 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:33.542319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0df63b-5d13-40fd-9c71-62151c612321 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:33.555045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5057c4-f9dc-4b5b-97e4-011bcb33650f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:33.556184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5057c4-f9dc-4b5b-97e4-011bcb33650f admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:33.558511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5057c4-f9dc-4b5b-97e4-011bcb33650f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:33.559920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5057c4-f9dc-4b5b-97e4-011bcb33650f admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:33.560918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5057c4-f9dc-4b5b-97e4-011bcb33650f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:33.563706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5057c4-f9dc-4b5b-97e4-011bcb33650f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:33.621992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 850/1694] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:33.626453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe4791-58f0-41ec-8514-8a391de3a07f admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:33.639336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe4791-58f0-41ec-8514-8a391de3a07f admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:33.661215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f30bac-1a11-4fe8-9377-0955dff2a0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:33.661852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f30bac-1a11-4fe8-9377-0955dff2a0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:33.662169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f30bac-1a11-4fe8-9377-0955dff2a0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:33.662442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f30bac-1a11-4fe8-9377-0955dff2a0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:33.663145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f30bac-1a11-4fe8-9377-0955dff2a0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:33.664743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f30bac-1a11-4fe8-9377-0955dff2a0f2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:33.709912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 845/1695] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:33.719009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5057c4-f9dc-4b5b-97e4-011bcb33650f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:33.746546 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc758-21f5-4c4c-8802-62ea9cd41fb9 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:33.747510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc758-21f5-4c4c-8802-62ea9cd41fb9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:33.747971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc758-21f5-4c4c-8802-62ea9cd41fb9 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:33.750700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 846/1696] 10.222.0.73 () {54 vars in 912 bytes} [Thu Aug 4 21:40:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:40:33.753607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5057c4-f9dc-4b5b-97e4-011bcb33650f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:33.756358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbb030-5b1d-4eb1-b064-ee7f69c188d3 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:33.756838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbb030-5b1d-4eb1-b064-ee7f69c188d3 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:33.757455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbb030-5b1d-4eb1-b064-ee7f69c188d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:33.767014 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:33.790734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5af4e-d31e-41ec-a518-7f91720bd962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:33.792335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5af4e-d31e-41ec-a518-7f91720bd962 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:33.794174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5af4e-d31e-41ec-a518-7f91720bd962 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:33.794634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5af4e-d31e-41ec-a518-7f91720bd962 admin admin] PATH_INFO: `/v3/projects/c7c8e17399fc42d5839bda667d2ebe75/users/8249aca8c5b74f54945f04094d348057/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:33.868506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5af4e-d31e-41ec-a518-7f91720bd962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c8e17399fc42d5839bda667d2ebe75, user_id=8249aca8c5b74f54945f04094d348057, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:33.894040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5af4e-d31e-41ec-a518-7f91720bd962 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:33.912158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-cedbb030-5b1d-4eb1-b064-ee7f69c188d3 None None] MFA Rules not processed for user `c98761f81496456fb3b5a5cffd4ce763`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:33.913885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-cedbb030-5b1d-4eb1-b064-ee7f69c188d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:34.006150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:34.007832 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 851/1697] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:33 2022] PUT /identity/v3/projects/c7c8e17399fc42d5839bda667d2ebe75/users/8249aca8c5b74f54945f04094d348057/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:34.020238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9884924-fe89-4e10-90a5-6cdad2f12b45 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:34.020955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9884924-fe89-4e10-90a5-6cdad2f12b45 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:34.021380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9884924-fe89-4e10-90a5-6cdad2f12b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:34.052803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 847/1698] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:40:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:34.054833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d603c6-6b47-4db0-85a2-3d7804032fcd None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:34.055490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d603c6-6b47-4db0-85a2-3d7804032fcd None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:34.055999 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d603c6-6b47-4db0-85a2-3d7804032fcd None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:34.059871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 848/1699] 10.222.0.73 () {54 vars in 912 bytes} [Thu Aug 4 21:40:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:40:34.108514 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb180c2-d2b4-4907-9d66-9246b1300eff None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:34.111066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb180c2-d2b4-4907-9d66-9246b1300eff None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:34.113236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb180c2-d2b4-4907-9d66-9246b1300eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:34.143508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d9884924-fe89-4e10-90a5-6cdad2f12b45 None None] MFA Rules not processed for user `8249aca8c5b74f54945f04094d348057`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:34.161124 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d9884924-fe89-4e10-90a5-6cdad2f12b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:34.183293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-edb180c2-d2b4-4907-9d66-9246b1300eff None None] MFA Rules not processed for user `c98761f81496456fb3b5a5cffd4ce763`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:34.186124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-edb180c2-d2b4-4907-9d66-9246b1300eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:34.318307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 849/1700] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:40:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:34.320660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb180c2-d2b4-4907-9d66-9246b1300eff None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:34.343562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb180c2-d2b4-4907-9d66-9246b1300eff None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:34.354686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f53975-578d-4caa-b02a-695d04ccc2d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:34.354903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f53975-578d-4caa-b02a-695d04ccc2d2 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:34.355074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f53975-578d-4caa-b02a-695d04ccc2d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:34.355544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f53975-578d-4caa-b02a-695d04ccc2d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:34.363557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 852/1701] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:34.367759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f53975-578d-4caa-b02a-695d04ccc2d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:34.368332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9884924-fe89-4e10-90a5-6cdad2f12b45 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:34.369201 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f53975-578d-4caa-b02a-695d04ccc2d2 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:34.379690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 850/1702] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:34.380513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f53975-578d-4caa-b02a-695d04ccc2d2 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:34.381957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-57f53975-578d-4caa-b02a-695d04ccc2d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:34.403896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f53975-578d-4caa-b02a-695d04ccc2d2 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:34.422558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26584e-4131-471b-bd5a-865e31dae8d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:34.424324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26584e-4131-471b-bd5a-865e31dae8d4 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:34.425692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26584e-4131-471b-bd5a-865e31dae8d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:34.426144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26584e-4131-471b-bd5a-865e31dae8d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:34.437133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26584e-4131-471b-bd5a-865e31dae8d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:34.440508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26584e-4131-471b-bd5a-865e31dae8d4 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:34.449819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 851/1703] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:34.450799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26584e-4131-471b-bd5a-865e31dae8d4 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:34.452787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-9b26584e-4131-471b-bd5a-865e31dae8d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:34.553345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9884924-fe89-4e10-90a5-6cdad2f12b45 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:34.565656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b44ae-da7f-441d-ac4a-532e98b41483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:34.566483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b44ae-da7f-441d-ac4a-532e98b41483 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:34.566864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b44ae-da7f-441d-ac4a-532e98b41483 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:34.567779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b44ae-da7f-441d-ac4a-532e98b41483 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:34.568586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b44ae-da7f-441d-ac4a-532e98b41483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:34.578456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b44ae-da7f-441d-ac4a-532e98b41483 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:34.583455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26584e-4131-471b-bd5a-865e31dae8d4 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:34.605195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:34.609188 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:34.613830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:34.613830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] PATH_INFO: `/v3/projects/df5356c0de064c2e83109c058870efb1` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:34.613830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5356c0de064c2e83109c058870efb1)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:34.613830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:34.749727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 853/1704] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:34 2022] POST /identity/v3/projects => generated 379 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:34.749727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b44ae-da7f-441d-ac4a-532e98b41483 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:34.749920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a17b44ae-da7f-441d-ac4a-532e98b41483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:34.834183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b44ae-da7f-441d-ac4a-532e98b41483 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:34.837181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5356c0de064c2e83109c058870efb1', 'request_id': 'req-b2c35a10-f352-4469-a9fc-2183d6f1b315'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:34.927914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5356c0de064c2e83109c058870efb1', 'request_id': 'req-b2c35a10-f352-4469-a9fc-2183d6f1b315'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:34.943347 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5356c0de064c2e83109c058870efb1', 'request_id': 'req-b2c35a10-f352-4469-a9fc-2183d6f1b315'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:34.946559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:34.950207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:34.950498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:34.950710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] PATH_INFO: `/v3/users/4263fdd2cf4e4a46a38adba3c41d1dae` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:34.957318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4263fdd2cf4e4a46a38adba3c41d1dae)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:34.960748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:34.975313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] The token cache is being invalidate because project df5356c0de064c2e83109c058870efb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:34.975615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:34.977879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b2c35a10-f352-4469-a9fc-2183d6f1b315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:34.987954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 852/1705] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:34 2022] DELETE /identity/v3/projects/df5356c0de064c2e83109c058870efb1 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:34.989323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72cc00-88ce-445f-a21e-64d65cacc5fa None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:34.989609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72cc00-88ce-445f-a21e-64d65cacc5fa None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:34.992472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72cc00-88ce-445f-a21e-64d65cacc5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:35.042203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-5f72cc00-88ce-445f-a21e-64d65cacc5fa None None] MFA Rules not processed for user `a23fcabad2354e7ab51a22b412bf11e6`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:35.044815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5f72cc00-88ce-445f-a21e-64d65cacc5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:35.128384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 853/1706] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:35.128384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72cc00-88ce-445f-a21e-64d65cacc5fa None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:35.134202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5f72cc00-88ce-445f-a21e-64d65cacc5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:35.144763 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72cc00-88ce-445f-a21e-64d65cacc5fa None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:35.155851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1adf9-cacb-47f0-9676-01f894051dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:35.156546 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1adf9-cacb-47f0-9676-01f894051dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:35.159430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1adf9-cacb-47f0-9676-01f894051dab admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:35.159430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1adf9-cacb-47f0-9676-01f894051dab admin admin] PATH_INFO: `/v3/users/9b916c33b04e494eb413f6832379bf0c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:35.159430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1adf9-cacb-47f0-9676-01f894051dab admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b916c33b04e494eb413f6832379bf0c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:35.160594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1adf9-cacb-47f0-9676-01f894051dab admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:35.258472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4263fdd2cf4e4a46a38adba3c41d1dae'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:35.373302 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4263fdd2cf4e4a46a38adba3c41d1dae'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:35.374290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4263fdd2cf4e4a46a38adba3c41d1dae'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:35.414446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4263fdd2cf4e4a46a38adba3c41d1dae'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:35.464758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4263fdd2cf4e4a46a38adba3c41d1dae'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:35.481473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 854/1707] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:34 2022] DELETE /identity/v3/users/4263fdd2cf4e4a46a38adba3c41d1dae => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:35.482259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:35.493365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:35.549074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b0d1adf9-cacb-47f0-9676-01f894051dab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b916c33b04e494eb413f6832379bf0c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:35.633511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2c756-8b83-4c25-a852-fa61a4dde276 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:35.642138 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06627aea-21b7-4315-8ed5-14655f9379dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:35.645002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b0d1adf9-cacb-47f0-9676-01f894051dab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b916c33b04e494eb413f6832379bf0c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:35.648760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06627aea-21b7-4315-8ed5-14655f9379dc admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:35.650245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06627aea-21b7-4315-8ed5-14655f9379dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:35.654934 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06627aea-21b7-4315-8ed5-14655f9379dc admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:35.657160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06627aea-21b7-4315-8ed5-14655f9379dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:35.662142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06627aea-21b7-4315-8ed5-14655f9379dc admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:35.670607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b0d1adf9-cacb-47f0-9676-01f894051dab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b916c33b04e494eb413f6832379bf0c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:35.671453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b0d1adf9-cacb-47f0-9676-01f894051dab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b916c33b04e494eb413f6832379bf0c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:35.691493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b0d1adf9-cacb-47f0-9676-01f894051dab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b916c33b04e494eb413f6832379bf0c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:35.715945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 854/1708] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:35 2022] DELETE /identity/v3/users/9b916c33b04e494eb413f6832379bf0c => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:35.715945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c80522-4254-4da6-83b4-36abe66eccc8 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:35.715945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c80522-4254-4da6-83b4-36abe66eccc8 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:35.715945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c80522-4254-4da6-83b4-36abe66eccc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:35.739061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-51c80522-4254-4da6-83b4-36abe66eccc8 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:35.745771 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-51c80522-4254-4da6-83b4-36abe66eccc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:35.806230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 855/1709] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:35 2022] POST /identity/v3/users => generated 296 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:35.813938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a839f-28f2-4ba6-a435-97b458cf447c None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:35.814658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a839f-28f2-4ba6-a435-97b458cf447c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:35.815122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a839f-28f2-4ba6-a435-97b458cf447c None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:35.821897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 856/1710] 10.222.0.73 () {54 vars in 961 bytes} [Thu Aug 4 21:40:35 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:40:35.823992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0483f-3f15-4cdf-8032-f37ae3004bd1 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:35.824478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0483f-3f15-4cdf-8032-f37ae3004bd1 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:35.826254 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0483f-3f15-4cdf-8032-f37ae3004bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:35.861505 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 855/1711] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:35.863255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c80522-4254-4da6-83b4-36abe66eccc8 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:35.873401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-51c80522-4254-4da6-83b4-36abe66eccc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:35.890686 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a4f0483f-3f15-4cdf-8032-f37ae3004bd1 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:35.895693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a4f0483f-3f15-4cdf-8032-f37ae3004bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:35.928035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 857/1712] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:35.931577 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2619868-b1f3-4737-9958-87db20578aec None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:35.931868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2619868-b1f3-4737-9958-87db20578aec None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:35.935922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2619868-b1f3-4737-9958-87db20578aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.003684 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d2619868-b1f3-4737-9958-87db20578aec None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:36.005147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d2619868-b1f3-4737-9958-87db20578aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:36.031944 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 858/1713] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:36.036099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc5b32-7562-49b4-971d-627b2d1713c2 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:36.036435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc5b32-7562-49b4-971d-627b2d1713c2 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:36.036724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc5b32-7562-49b4-971d-627b2d1713c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.054977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c80522-4254-4da6-83b4-36abe66eccc8 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:36.075692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6ba6d-6c67-4906-ae1e-38bfd780e6c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:36.076500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6ba6d-6c67-4906-ae1e-38bfd780e6c9 service placement] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:36.077044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6ba6d-6c67-4906-ae1e-38bfd780e6c9 service placement] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:36.077439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6ba6d-6c67-4906-ae1e-38bfd780e6c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.079489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-06f6ba6d-6c67-4906-ae1e-38bfd780e6c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:36.097392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-b5dc5b32-7562-49b4-971d-627b2d1713c2 None None] MFA Rules not processed for user `d9bc42bec393430b9b3e9399a8c4f95e`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:36.098283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b5dc5b32-7562-49b4-971d-627b2d1713c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:36.101069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6ba6d-6c67-4906-ae1e-38bfd780e6c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:36.104677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6ba6d-6c67-4906-ae1e-38bfd780e6c9 service placement] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:36.124716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 856/1714] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:36.134964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6ba6d-6c67-4906-ae1e-38bfd780e6c9 service placement] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:36.136714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-06f6ba6d-6c67-4906-ae1e-38bfd780e6c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:36.165238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6ba6d-6c67-4906-ae1e-38bfd780e6c9 service placement] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:36.181138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345287e-cd5b-405d-b898-a641d070ecd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:36.181914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345287e-cd5b-405d-b898-a641d070ecd4 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:36.182406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345287e-cd5b-405d-b898-a641d070ecd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:36.182783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345287e-cd5b-405d-b898-a641d070ecd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.183751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345287e-cd5b-405d-b898-a641d070ecd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:36.186178 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345287e-cd5b-405d-b898-a641d070ecd4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:36.217803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 857/1715] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:36.219620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b11f2-ddcd-4dac-a2cb-140be14274f4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:36.222541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b11f2-ddcd-4dac-a2cb-140be14274f4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:36.222928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b11f2-ddcd-4dac-a2cb-140be14274f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.226878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:36.267426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 859/1716] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:36.270516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744670-ad7a-4662-8ead-2eb05c0a3943 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:36.274750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744670-ad7a-4662-8ead-2eb05c0a3943 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:36.274969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744670-ad7a-4662-8ead-2eb05c0a3943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.305176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-062b11f2-ddcd-4dac-a2cb-140be14274f4 None None] MFA Rules not processed for user `fab36b186fa44ee18bf72228fc7ba758`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:36.308729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-062b11f2-ddcd-4dac-a2cb-140be14274f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:36.314562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-26744670-ad7a-4662-8ead-2eb05c0a3943 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:36.316045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-26744670-ad7a-4662-8ead-2eb05c0a3943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:36.334197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 858/1717] 10.222.0.73 () {58 vars in 1058 bytes} [Thu Aug 4 21:40:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:36.344479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b11f2-ddcd-4dac-a2cb-140be14274f4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:36.347310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 860/1718] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:36.348245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-062b11f2-ddcd-4dac-a2cb-140be14274f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:36.358723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26744670-ad7a-4662-8ead-2eb05c0a3943 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:36.362654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-26744670-ad7a-4662-8ead-2eb05c0a3943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:36.470145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:36.531374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26744670-ad7a-4662-8ead-2eb05c0a3943 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:36.545705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551710be-b793-4b74-9b3f-d03beb3366d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:36.546775 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551710be-b793-4b74-9b3f-d03beb3366d0 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:36.547201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551710be-b793-4b74-9b3f-d03beb3366d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:36.554695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551710be-b793-4b74-9b3f-d03beb3366d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.565862 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b11f2-ddcd-4dac-a2cb-140be14274f4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:36.571777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c76ce-1805-462f-9d2a-a13b02022a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:36.572426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c76ce-1805-462f-9d2a-a13b02022a0c service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:36.572732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c76ce-1805-462f-9d2a-a13b02022a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:36.573617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c76ce-1805-462f-9d2a-a13b02022a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.573918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551710be-b793-4b74-9b3f-d03beb3366d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:36.575565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551710be-b793-4b74-9b3f-d03beb3366d0 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:36.597321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c76ce-1805-462f-9d2a-a13b02022a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:36.599594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c76ce-1805-462f-9d2a-a13b02022a0c service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:36.601764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 861/1719] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:36.608310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551710be-b793-4b74-9b3f-d03beb3366d0 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:36.635137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551710be-b793-4b74-9b3f-d03beb3366d0 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:36.641628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c569900-f9dd-4a2a-a103-774fb7adfc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:36.642656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c569900-f9dd-4a2a-a103-774fb7adfc4e admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:36.643158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c569900-f9dd-4a2a-a103-774fb7adfc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:36.643580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c569900-f9dd-4a2a-a103-774fb7adfc4e admin admin] PATH_INFO: `/v3/projects/55613d013a2d4e2f9b30d9d42a16d671/users/3398dd480985445fbcd08573d105958d/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.646583 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 859/1720] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:36.647639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c76ce-1805-462f-9d2a-a13b02022a0c service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:36.690308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c569900-f9dd-4a2a-a103-774fb7adfc4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55613d013a2d4e2f9b30d9d42a16d671, user_id=3398dd480985445fbcd08573d105958d, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:36.692712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c76ce-1805-462f-9d2a-a13b02022a0c service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:36.693622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c569900-f9dd-4a2a-a103-774fb7adfc4e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:36.705916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daea9a1-e807-4d5c-a701-1ecfefd45a0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:36.707205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daea9a1-e807-4d5c-a701-1ecfefd45a0e service placement] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:36.707205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daea9a1-e807-4d5c-a701-1ecfefd45a0e service placement] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:36.707205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daea9a1-e807-4d5c-a701-1ecfefd45a0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.720095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daea9a1-e807-4d5c-a701-1ecfefd45a0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:36.721467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daea9a1-e807-4d5c-a701-1ecfefd45a0e service placement] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:36.732648 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 860/1721] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:36.734097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daea9a1-e807-4d5c-a701-1ecfefd45a0e service placement] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:36.744215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daea9a1-e807-4d5c-a701-1ecfefd45a0e service placement] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:36.754278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13063d1-807b-4b94-8d20-1cafca561da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:36.754536 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13063d1-807b-4b94-8d20-1cafca561da7 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:36.754939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13063d1-807b-4b94-8d20-1cafca561da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:36.755794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13063d1-807b-4b94-8d20-1cafca561da7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.756034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13063d1-807b-4b94-8d20-1cafca561da7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:36.756401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13063d1-807b-4b94-8d20-1cafca561da7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:36.758130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13063d1-807b-4b94-8d20-1cafca561da7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:36.778767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 861/1722] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:36.784160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13063d1-807b-4b94-8d20-1cafca561da7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:36.798629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13063d1-807b-4b94-8d20-1cafca561da7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:36.803879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa335ac-8ec4-4949-868a-34ea9ae4816e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:36.804511 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa335ac-8ec4-4949-868a-34ea9ae4816e admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:36.804995 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa335ac-8ec4-4949-868a-34ea9ae4816e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:36.805339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa335ac-8ec4-4949-868a-34ea9ae4816e admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:36.806601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa335ac-8ec4-4949-868a-34ea9ae4816e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:36.809934 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa335ac-8ec4-4949-868a-34ea9ae4816e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:36.831796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 862/1723] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:36 2022] PUT /identity/v3/projects/55613d013a2d4e2f9b30d9d42a16d671/users/3398dd480985445fbcd08573d105958d/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:36.835354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c569900-f9dd-4a2a-a103-774fb7adfc4e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:36.997851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c569900-f9dd-4a2a-a103-774fb7adfc4e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:37.003393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 862/1724] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:36 2022] POST /identity/v3/projects => generated 371 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:37.007401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa335ac-8ec4-4949-868a-34ea9ae4816e admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:37.224000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa335ac-8ec4-4949-868a-34ea9ae4816e admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:37.248578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf7189-596f-47c5-92a4-b1b4ba9a51c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:37.249362 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed12da-f1ac-4d78-8931-334de499e4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:37.250589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7189-596f-47c5-92a4-b1b4ba9a51c2 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:37.253145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed12da-f1ac-4d78-8931-334de499e4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:37.253648 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed12da-f1ac-4d78-8931-334de499e4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:37.254181 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7189-596f-47c5-92a4-b1b4ba9a51c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:37.254982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed12da-f1ac-4d78-8931-334de499e4d1 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:37.256273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed12da-f1ac-4d78-8931-334de499e4d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:37.256485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7189-596f-47c5-92a4-b1b4ba9a51c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:37.257350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf7189-596f-47c5-92a4-b1b4ba9a51c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:37.262028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf7189-596f-47c5-92a4-b1b4ba9a51c2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:37.263382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed12da-f1ac-4d78-8931-334de499e4d1 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:37.293596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 863/1725] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:37.296234 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf7189-596f-47c5-92a4-b1b4ba9a51c2 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:37.325449 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf7189-596f-47c5-92a4-b1b4ba9a51c2 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:37.330332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109fb8de-6804-4aeb-97a9-f52e343147f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:37.331298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109fb8de-6804-4aeb-97a9-f52e343147f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:37.331785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109fb8de-6804-4aeb-97a9-f52e343147f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:37.332261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109fb8de-6804-4aeb-97a9-f52e343147f4 admin admin] PATH_INFO: `/v3/projects/55613d013a2d4e2f9b30d9d42a16d671/users/3398dd480985445fbcd08573d105958d/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:37.334841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109fb8de-6804-4aeb-97a9-f52e343147f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55613d013a2d4e2f9b30d9d42a16d671, user_id=3398dd480985445fbcd08573d105958d, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:37.353341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109fb8de-6804-4aeb-97a9-f52e343147f4 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:37.377315 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 864/1726] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:37 2022] PUT /identity/v3/projects/55613d013a2d4e2f9b30d9d42a16d671/users/3398dd480985445fbcd08573d105958d/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:37.388493 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf64cc-7346-40c2-989b-a1d86fb81490 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:37.388735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf64cc-7346-40c2-989b-a1d86fb81490 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:37.389017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf64cc-7346-40c2-989b-a1d86fb81490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:37.402097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 863/1727] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:37 2022] POST /identity/v3/users => generated 286 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:37.420984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed12da-f1ac-4d78-8931-334de499e4d1 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:37.434702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-f8bf64cc-7346-40c2-989b-a1d86fb81490 None None] MFA Rules not processed for user `3398dd480985445fbcd08573d105958d`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:37.437674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f8bf64cc-7346-40c2-989b-a1d86fb81490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:37.447050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed12da-f1ac-4d78-8931-334de499e4d1 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:37.547679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3921cb7-5b49-46ae-bd2e-d73c30bc71c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:37.548540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3921cb7-5b49-46ae-bd2e-d73c30bc71c2 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:37.548914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3921cb7-5b49-46ae-bd2e-d73c30bc71c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:37.549433 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3921cb7-5b49-46ae-bd2e-d73c30bc71c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:37.550286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3921cb7-5b49-46ae-bd2e-d73c30bc71c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:37.552198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 865/1728] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:37.552672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3921cb7-5b49-46ae-bd2e-d73c30bc71c2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:37.564982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf64cc-7346-40c2-989b-a1d86fb81490 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:37.568436 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f8bf64cc-7346-40c2-989b-a1d86fb81490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:37.578843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 864/1729] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:37.579960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3921cb7-5b49-46ae-bd2e-d73c30bc71c2 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:37.592264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3921cb7-5b49-46ae-bd2e-d73c30bc71c2 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:37.599064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d7974-cce1-4a77-99a5-b5eb8f648fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:37.600121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d7974-cce1-4a77-99a5-b5eb8f648fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:37.600613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d7974-cce1-4a77-99a5-b5eb8f648fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:37.601050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d7974-cce1-4a77-99a5-b5eb8f648fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:37.601969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d7974-cce1-4a77-99a5-b5eb8f648fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:37.603753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d7974-cce1-4a77-99a5-b5eb8f648fb9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:37.616036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 865/1730] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:37.621028 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d7974-cce1-4a77-99a5-b5eb8f648fb9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:37.645509 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d7974-cce1-4a77-99a5-b5eb8f648fb9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:37.658469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64046244-bb12-4a22-8c1d-f325f3c89948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:37.660099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046244-bb12-4a22-8c1d-f325f3c89948 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:37.660473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046244-bb12-4a22-8c1d-f325f3c89948 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:37.661812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046244-bb12-4a22-8c1d-f325f3c89948 admin admin] PATH_INFO: `/v3/projects/b64126c6313746c09e07f06369c64711/users/36ef8cd66e3443979b2f312cb680a362/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:37.666212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf64cc-7346-40c2-989b-a1d86fb81490 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:37.671011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8bd912-439b-484e-8a30-b7edd241121d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:37.671648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bd912-439b-484e-8a30-b7edd241121d service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:37.671983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bd912-439b-484e-8a30-b7edd241121d service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:37.672256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bd912-439b-484e-8a30-b7edd241121d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:37.682763 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64046244-bb12-4a22-8c1d-f325f3c89948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b64126c6313746c09e07f06369c64711, user_id=36ef8cd66e3443979b2f312cb680a362, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:37.682954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8bd912-439b-484e-8a30-b7edd241121d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:37.684760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8bd912-439b-484e-8a30-b7edd241121d service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:37.685928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64046244-bb12-4a22-8c1d-f325f3c89948 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:37.706965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 866/1731] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:37.780342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:37.849119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 866/1732] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:37 2022] PUT /identity/v3/projects/b64126c6313746c09e07f06369c64711/users/36ef8cd66e3443979b2f312cb680a362/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:37.866850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c9038-dd5b-4cd3-ac81-8674f955a5c3 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:37.867072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c9038-dd5b-4cd3-ac81-8674f955a5c3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:37.867294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c9038-dd5b-4cd3-ac81-8674f955a5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:37.918281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:37.944667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-dc9c9038-dd5b-4cd3-ac81-8674f955a5c3 None None] MFA Rules not processed for user `36ef8cd66e3443979b2f312cb680a362`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:37.946950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-dc9c9038-dd5b-4cd3-ac81-8674f955a5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:38.052127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64046244-bb12-4a22-8c1d-f325f3c89948 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:38.072692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 867/1733] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:38.072967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c9038-dd5b-4cd3-ac81-8674f955a5c3 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:38.076324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-dc9c9038-dd5b-4cd3-ac81-8674f955a5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:38.218874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64046244-bb12-4a22-8c1d-f325f3c89948 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:38.223342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb820db7-a211-470b-b7b9-e73e7cc8a9a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:38.224257 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb820db7-a211-470b-b7b9-e73e7cc8a9a8 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:38.225266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb820db7-a211-470b-b7b9-e73e7cc8a9a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:38.225744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb820db7-a211-470b-b7b9-e73e7cc8a9a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:38.230628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bb820db7-a211-470b-b7b9-e73e7cc8a9a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:38.256103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c9038-dd5b-4cd3-ac81-8674f955a5c3 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:38.265863 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:38.266669 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:38.267013 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:38.267332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] PATH_INFO: `/v3/projects/38baafa2203044b9bec92dc99e18d36c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:38.268628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38baafa2203044b9bec92dc99e18d36c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:38.271137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:38.322997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb820db7-a211-470b-b7b9-e73e7cc8a9a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:38.324770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb820db7-a211-470b-b7b9-e73e7cc8a9a8 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:38.343808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 867/1734] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:40:38.344802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb820db7-a211-470b-b7b9-e73e7cc8a9a8 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:38.377936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb820db7-a211-470b-b7b9-e73e7cc8a9a8 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:38.398832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95797205-24ea-4ff2-afc3-b0222e3c620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:38.399636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95797205-24ea-4ff2-afc3-b0222e3c620a admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:38.400366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95797205-24ea-4ff2-afc3-b0222e3c620a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:38.401883 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95797205-24ea-4ff2-afc3-b0222e3c620a admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:38.402747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95797205-24ea-4ff2-afc3-b0222e3c620a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:38.411039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95797205-24ea-4ff2-afc3-b0222e3c620a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:38.440858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38baafa2203044b9bec92dc99e18d36c', 'request_id': 'req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:38.462737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38baafa2203044b9bec92dc99e18d36c', 'request_id': 'req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:38.607608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38baafa2203044b9bec92dc99e18d36c', 'request_id': 'req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:38.629495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 868/1735] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:38 2022] POST /identity/v3/projects => generated 371 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:38.635027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95797205-24ea-4ff2-afc3-b0222e3c620a admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:38.639289 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-95797205-24ea-4ff2-afc3-b0222e3c620a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:38.645697 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] The token cache is being invalidate because project 38baafa2203044b9bec92dc99e18d36c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:38.646185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:38.647452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:38.656548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 868/1736] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:38 2022] DELETE /identity/v3/projects/38baafa2203044b9bec92dc99e18d36c => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:38.656548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:38.661401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:38.784265 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6a8a1-81bd-4cca-b01c-b6b60a539335 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:38.784548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95797205-24ea-4ff2-afc3-b0222e3c620a admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:38.789727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c4bb3-9a3b-4809-817e-c98657da225a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:38.790451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c4bb3-9a3b-4809-817e-c98657da225a admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:38.790737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c4bb3-9a3b-4809-817e-c98657da225a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:38.791014 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c4bb3-9a3b-4809-817e-c98657da225a admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:38.792052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:38.792308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c4bb3-9a3b-4809-817e-c98657da225a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:38.794638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c4bb3-9a3b-4809-817e-c98657da225a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:38.795879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:38.796572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:38.796572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] PATH_INFO: `/v3/projects/7b6134537ca84005a94c5e82b11277fd` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:38.803222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6134537ca84005a94c5e82b11277fd)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:38.808257 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:38.894601 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 869/1737] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:38 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:38.895755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c4bb3-9a3b-4809-817e-c98657da225a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:38.901733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ac9c4bb3-9a3b-4809-817e-c98657da225a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:38.917424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c4bb3-9a3b-4809-817e-c98657da225a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:38.923285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:38.924004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:38.924324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:38.924636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] PATH_INFO: `/v3/users/1e1fe30eafce48278f3d267ecd08fe0a` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:38.925886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e1fe30eafce48278f3d267ecd08fe0a)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:38.928074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:38.970451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6134537ca84005a94c5e82b11277fd', 'request_id': 'req-24368503-f450-428c-b9c4-6f3291b0a440'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:39.075549 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6134537ca84005a94c5e82b11277fd', 'request_id': 'req-24368503-f450-428c-b9c4-6f3291b0a440'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:39.081765 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6134537ca84005a94c5e82b11277fd', 'request_id': 'req-24368503-f450-428c-b9c4-6f3291b0a440'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:39.093921 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] The token cache is being invalidate because project 7b6134537ca84005a94c5e82b11277fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:39.094294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:39.095328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24368503-f450-428c-b9c4-6f3291b0a440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:39.097315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 869/1738] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:38 2022] DELETE /identity/v3/projects/7b6134537ca84005a94c5e82b11277fd => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:39.100596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f2e42-f466-46be-b86b-ceead764af8f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:39.100596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f2e42-f466-46be-b86b-ceead764af8f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:39.100596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f2e42-f466-46be-b86b-ceead764af8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:39.122907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c67f2e42-f466-46be-b86b-ceead764af8f None None] MFA Rules not processed for user `95fb2edf82644e7f8ce461fcbb52007b`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:39.124217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c67f2e42-f466-46be-b86b-ceead764af8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:39.192513 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e1fe30eafce48278f3d267ecd08fe0a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:39.213100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 870/1739] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:39.213100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f2e42-f466-46be-b86b-ceead764af8f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:39.213100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c67f2e42-f466-46be-b86b-ceead764af8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:39.297392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e1fe30eafce48278f3d267ecd08fe0a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:39.300943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e1fe30eafce48278f3d267ecd08fe0a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:39.334522 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f2e42-f466-46be-b86b-ceead764af8f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:39.345005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e06d88-2e34-4fe5-b323-b03fbb317277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:39.345857 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e06d88-2e34-4fe5-b323-b03fbb317277 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:39.346322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e06d88-2e34-4fe5-b323-b03fbb317277 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:39.346736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e06d88-2e34-4fe5-b323-b03fbb317277 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:39.347610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e06d88-2e34-4fe5-b323-b03fbb317277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:39.351835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e06d88-2e34-4fe5-b323-b03fbb317277 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:39.361268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e1fe30eafce48278f3d267ecd08fe0a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:39.368593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 871/1740] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:39.370915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28801a0-7dd3-44be-9e73-0d80432c837f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:39.372000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28801a0-7dd3-44be-9e73-0d80432c837f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:39.373570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28801a0-7dd3-44be-9e73-0d80432c837f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:39.380737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e1fe30eafce48278f3d267ecd08fe0a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:39.401308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-a28801a0-7dd3-44be-9e73-0d80432c837f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:39.401308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a28801a0-7dd3-44be-9e73-0d80432c837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:39.409413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 870/1741] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:38 2022] DELETE /identity/v3/users/1e1fe30eafce48278f3d267ecd08fe0a => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:39.413799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:39.417908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:39.437740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 872/1742] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:39.439482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ef424-2806-476e-98ec-e77de1e22895 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:39.439776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ef424-2806-476e-98ec-e77de1e22895 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:39.440051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ef424-2806-476e-98ec-e77de1e22895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:39.493795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-379ef424-2806-476e-98ec-e77de1e22895 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:39.498441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-379ef424-2806-476e-98ec-e77de1e22895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:39.627322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb903460-4798-4aa1-bef8-da9232ace13a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:39.647899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 873/1743] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:39.652001 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ef424-2806-476e-98ec-e77de1e22895 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:39.657193 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87d92d-3b4d-478d-a682-d722ec8f803c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:39.657542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87d92d-3b4d-478d-a682-d722ec8f803c admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:39.657845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87d92d-3b4d-478d-a682-d722ec8f803c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:39.658765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87d92d-3b4d-478d-a682-d722ec8f803c admin admin] PATH_INFO: `/v3/projects/fe49d009c94d431ea12286d8754fd383/users/4f1d27ebd7474643bfa2543ef1af3bf1/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:39.668361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ef424-2806-476e-98ec-e77de1e22895 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:39.684512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7dba2-d8a8-4f7b-8b74-c90789369679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:39.685284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7dba2-d8a8-4f7b-8b74-c90789369679 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:39.691781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7dba2-d8a8-4f7b-8b74-c90789369679 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:39.691781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7dba2-d8a8-4f7b-8b74-c90789369679 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:39.691781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7dba2-d8a8-4f7b-8b74-c90789369679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:39.691781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7dba2-d8a8-4f7b-8b74-c90789369679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:39.691781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7dba2-d8a8-4f7b-8b74-c90789369679 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:39.704336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 874/1744] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:39.709078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87d92d-3b4d-478d-a682-d722ec8f803c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe49d009c94d431ea12286d8754fd383, user_id=4f1d27ebd7474643bfa2543ef1af3bf1, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:39.709250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7dba2-d8a8-4f7b-8b74-c90789369679 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:39.711725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87d92d-3b4d-478d-a682-d722ec8f803c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:39.724524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7dba2-d8a8-4f7b-8b74-c90789369679 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:39.730630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fac9a-253a-4c3e-9030-943550cd5bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:39.731383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fac9a-253a-4c3e-9030-943550cd5bfe admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:39.732172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fac9a-253a-4c3e-9030-943550cd5bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:39.732172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fac9a-253a-4c3e-9030-943550cd5bfe admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:39.733128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fac9a-253a-4c3e-9030-943550cd5bfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:39.735567 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fac9a-253a-4c3e-9030-943550cd5bfe admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:39.829752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 871/1745] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:39 2022] PUT /identity/v3/projects/fe49d009c94d431ea12286d8754fd383/users/4f1d27ebd7474643bfa2543ef1af3bf1/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:39.835177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87d92d-3b4d-478d-a682-d722ec8f803c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:39.888933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 875/1746] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:39 2022] POST /identity/v3/projects => generated 369 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:39.893369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fac9a-253a-4c3e-9030-943550cd5bfe admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:39.951845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87d92d-3b4d-478d-a682-d722ec8f803c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:40.028260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fac9a-253a-4c3e-9030-943550cd5bfe admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:40.040153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbc470-fc10-41be-a36c-21ea8c07ac2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:40.042047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc470-fc10-41be-a36c-21ea8c07ac2b admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:40.042600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc470-fc10-41be-a36c-21ea8c07ac2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:40.043020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc470-fc10-41be-a36c-21ea8c07ac2b admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:40.043914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbc470-fc10-41be-a36c-21ea8c07ac2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:40.044031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02b4b3-3268-4d44-b985-039b44b7ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:40.045638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02b4b3-3268-4d44-b985-039b44b7ddad admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:40.045935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02b4b3-3268-4d44-b985-039b44b7ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:40.046810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbc470-fc10-41be-a36c-21ea8c07ac2b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:40.047426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02b4b3-3268-4d44-b985-039b44b7ddad admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:40.049355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02b4b3-3268-4d44-b985-039b44b7ddad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:40.055645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02b4b3-3268-4d44-b985-039b44b7ddad admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:40.069117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 872/1747] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:40.073179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbc470-fc10-41be-a36c-21ea8c07ac2b admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:40.110445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbc470-fc10-41be-a36c-21ea8c07ac2b admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:40.122222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105f45f-e59d-4b09-8e8b-9f9565b3752b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:40.123277 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105f45f-e59d-4b09-8e8b-9f9565b3752b admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:40.123882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105f45f-e59d-4b09-8e8b-9f9565b3752b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:40.124360 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105f45f-e59d-4b09-8e8b-9f9565b3752b admin admin] PATH_INFO: `/v3/projects/fe49d009c94d431ea12286d8754fd383/users/4f1d27ebd7474643bfa2543ef1af3bf1/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:40.127264 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105f45f-e59d-4b09-8e8b-9f9565b3752b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe49d009c94d431ea12286d8754fd383, user_id=4f1d27ebd7474643bfa2543ef1af3bf1, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:40.134695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105f45f-e59d-4b09-8e8b-9f9565b3752b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:40.149774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 873/1748] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:40 2022] PUT /identity/v3/projects/fe49d009c94d431ea12286d8754fd383/users/4f1d27ebd7474643bfa2543ef1af3bf1/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:40.164599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a2c39-ac5a-4b3e-abd9-599fb2b6f568 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:40.169263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a2c39-ac5a-4b3e-abd9-599fb2b6f568 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:40.169263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a2c39-ac5a-4b3e-abd9-599fb2b6f568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:40.233594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 876/1749] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:39 2022] POST /identity/v3/users => generated 285 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:40.239866 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02b4b3-3268-4d44-b985-039b44b7ddad admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:40.264595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02b4b3-3268-4d44-b985-039b44b7ddad admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:40.266550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-079a2c39-ac5a-4b3e-abd9-599fb2b6f568 None None] MFA Rules not processed for user `4f1d27ebd7474643bfa2543ef1af3bf1`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:40.272114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-079a2c39-ac5a-4b3e-abd9-599fb2b6f568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:40.658025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee4dcc-5e31-4592-b5a7-9df3f7764045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:40.658025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee4dcc-5e31-4592-b5a7-9df3f7764045 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:40.658025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee4dcc-5e31-4592-b5a7-9df3f7764045 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:40.658025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee4dcc-5e31-4592-b5a7-9df3f7764045 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:40.660979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee4dcc-5e31-4592-b5a7-9df3f7764045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:40.660979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee4dcc-5e31-4592-b5a7-9df3f7764045 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:40.660979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 877/1750] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:40:40.660979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee4dcc-5e31-4592-b5a7-9df3f7764045 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:40.660979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee4dcc-5e31-4592-b5a7-9df3f7764045 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:40.661374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537c1b1-5ec3-4803-8866-3925cb22dfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:40.661374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c1b1-5ec3-4803-8866-3925cb22dfa9 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:40.661374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c1b1-5ec3-4803-8866-3925cb22dfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:40.661374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c1b1-5ec3-4803-8866-3925cb22dfa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:40.661689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537c1b1-5ec3-4803-8866-3925cb22dfa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:40.661689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537c1b1-5ec3-4803-8866-3925cb22dfa9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:40.661837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 874/1751] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:40.662252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 878/1752] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:40:40.662252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537c1b1-5ec3-4803-8866-3925cb22dfa9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:40.662410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a2c39-ac5a-4b3e-abd9-599fb2b6f568 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:40.662410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-079a2c39-ac5a-4b3e-abd9-599fb2b6f568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:40.662617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537c1b1-5ec3-4803-8866-3925cb22dfa9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:40.662617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dca011-8183-4a53-b078-3806ed02ca55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:40.662617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dca011-8183-4a53-b078-3806ed02ca55 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:40.662617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dca011-8183-4a53-b078-3806ed02ca55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:40.662929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dca011-8183-4a53-b078-3806ed02ca55 admin admin] PATH_INFO: `/v3/projects/2e86dc866a794e1288aa71b83556693f/users/e7bf83d042634f6abd5968c9b36de102/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:40.662929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dca011-8183-4a53-b078-3806ed02ca55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e86dc866a794e1288aa71b83556693f, user_id=e7bf83d042634f6abd5968c9b36de102, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:40.662929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dca011-8183-4a53-b078-3806ed02ca55 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:40.663152 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a2c39-ac5a-4b3e-abd9-599fb2b6f568 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:40.663152 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bbedb-4489-417a-9c37-2660b16d1d3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:40.663152 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bbedb-4489-417a-9c37-2660b16d1d3d service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:40.663152 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bbedb-4489-417a-9c37-2660b16d1d3d service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:40.663484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bbedb-4489-417a-9c37-2660b16d1d3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:40.663484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732bbedb-4489-417a-9c37-2660b16d1d3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:40.663484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732bbedb-4489-417a-9c37-2660b16d1d3d service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:40.663484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 875/1753] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:40.893453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 879/1754] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:40 2022] PUT /identity/v3/projects/2e86dc866a794e1288aa71b83556693f/users/e7bf83d042634f6abd5968c9b36de102/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:40.908427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afae34-d978-4b17-97ea-3e2ccfd906f0 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:40.908956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afae34-d978-4b17-97ea-3e2ccfd906f0 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:40.909414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afae34-d978-4b17-97ea-3e2ccfd906f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:40.913350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dca011-8183-4a53-b078-3806ed02ca55 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:40.915171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f7dca011-8183-4a53-b078-3806ed02ca55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:40.991820 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-92afae34-d978-4b17-97ea-3e2ccfd906f0 None None] MFA Rules not processed for user `e7bf83d042634f6abd5968c9b36de102`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:40.997905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-92afae34-d978-4b17-97ea-3e2ccfd906f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:41.053997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dca011-8183-4a53-b078-3806ed02ca55 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:41.076268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:41.078431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:41.080557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:41.080557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] PATH_INFO: `/v3/projects/61cc92d7a4ad483a9970cf5b502df467` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:41.081613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61cc92d7a4ad483a9970cf5b502df467)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:41.083320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:41.084112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:41.114346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 876/1755] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:41.119230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afae34-d978-4b17-97ea-3e2ccfd906f0 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:41.156676 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afae34-d978-4b17-97ea-3e2ccfd906f0 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:41.165350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:41.173428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6435f-7740-4bf1-8748-0663d493e3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:41.173428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6435f-7740-4bf1-8748-0663d493e3ed admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:41.173428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6435f-7740-4bf1-8748-0663d493e3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:41.173428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6435f-7740-4bf1-8748-0663d493e3ed admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:41.173758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6435f-7740-4bf1-8748-0663d493e3ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:41.178257 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6435f-7740-4bf1-8748-0663d493e3ed admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:41.284342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61cc92d7a4ad483a9970cf5b502df467', 'request_id': 'req-48a968c8-e928-4939-a20f-683b454e3d95'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:41.347172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 877/1756] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:41 2022] POST /identity/v3/projects => generated 367 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:41.351268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6435f-7740-4bf1-8748-0663d493e3ed admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:41.371615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61cc92d7a4ad483a9970cf5b502df467', 'request_id': 'req-48a968c8-e928-4939-a20f-683b454e3d95'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:41.380029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61cc92d7a4ad483a9970cf5b502df467', 'request_id': 'req-48a968c8-e928-4939-a20f-683b454e3d95'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:41.392681 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] The token cache is being invalidate because project 61cc92d7a4ad483a9970cf5b502df467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:41.392975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:41.393696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:41.395254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 880/1757] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:40 2022] DELETE /identity/v3/projects/61cc92d7a4ad483a9970cf5b502df467 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:41.395992 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:41.397292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:41.414713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6435f-7740-4bf1-8748-0663d493e3ed admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:41.419262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5580700-a5e1-4d5f-a23e-5e87054af176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:41.419899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5580700-a5e1-4d5f-a23e-5e87054af176 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:41.420194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5580700-a5e1-4d5f-a23e-5e87054af176 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:41.420459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5580700-a5e1-4d5f-a23e-5e87054af176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:41.422474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a5580700-a5e1-4d5f-a23e-5e87054af176 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:41.479195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a968c8-e928-4939-a20f-683b454e3d95 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:41.491636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5580700-a5e1-4d5f-a23e-5e87054af176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:41.493293 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5580700-a5e1-4d5f-a23e-5e87054af176 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:41.533451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 878/1758] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:41.535228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c58c4-d440-4c42-b909-4cd44c36df68 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:41.535521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c58c4-d440-4c42-b909-4cd44c36df68 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:41.535816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c58c4-d440-4c42-b909-4cd44c36df68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:41.579844 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ca3c58c4-d440-4c42-b909-4cd44c36df68 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:41.583177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673ec64-5604-4dde-b7a5-d3deb05e950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:41.584175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673ec64-5604-4dde-b7a5-d3deb05e950e admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:41.584702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673ec64-5604-4dde-b7a5-d3deb05e950e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:41.585216 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673ec64-5604-4dde-b7a5-d3deb05e950e admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:41.586173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ca3c58c4-d440-4c42-b909-4cd44c36df68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:41.589625 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673ec64-5604-4dde-b7a5-d3deb05e950e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:41.591120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673ec64-5604-4dde-b7a5-d3deb05e950e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:41.607662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 879/1759] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:41.617089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ae3d8-dd75-424c-a4bd-41a2fce07349 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:41.622609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ae3d8-dd75-424c-a4bd-41a2fce07349 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:41.622609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ae3d8-dd75-424c-a4bd-41a2fce07349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:41.669967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-729ae3d8-dd75-424c-a4bd-41a2fce07349 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:41.672952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-729ae3d8-dd75-424c-a4bd-41a2fce07349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:41.728042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 880/1760] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:41.743488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 881/1761] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:41 2022] POST /identity/v3/users => generated 290 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:41.745938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ae3d8-dd75-424c-a4bd-41a2fce07349 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:41.747248 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673ec64-5604-4dde-b7a5-d3deb05e950e admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:41.771597 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673ec64-5604-4dde-b7a5-d3deb05e950e admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:41.772571 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ae3d8-dd75-424c-a4bd-41a2fce07349 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:41.783814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719722b9-2a9e-4c94-b2f1-a1c59d763abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:41.784531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719722b9-2a9e-4c94-b2f1-a1c59d763abb admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:41.784821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719722b9-2a9e-4c94-b2f1-a1c59d763abb admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:41.785110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719722b9-2a9e-4c94-b2f1-a1c59d763abb admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:41.785871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719722b9-2a9e-4c94-b2f1-a1c59d763abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:41.792604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37d824-fb21-4d72-9f53-40c4113c2dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:41.797980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37d824-fb21-4d72-9f53-40c4113c2dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:41.797980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37d824-fb21-4d72-9f53-40c4113c2dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:41.798602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719722b9-2a9e-4c94-b2f1-a1c59d763abb admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:41.798703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37d824-fb21-4d72-9f53-40c4113c2dc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:41.798703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37d824-fb21-4d72-9f53-40c4113c2dc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:41.798703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37d824-fb21-4d72-9f53-40c4113c2dc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:41.800808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37d824-fb21-4d72-9f53-40c4113c2dc2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:41.819504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 881/1762] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:41.824663 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37d824-fb21-4d72-9f53-40c4113c2dc2 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:41.830448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 882/1763] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:41.831394 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719722b9-2a9e-4c94-b2f1-a1c59d763abb admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:41.846392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37d824-fb21-4d72-9f53-40c4113c2dc2 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:41.849963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719722b9-2a9e-4c94-b2f1-a1c59d763abb admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:41.856350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586d367-3c15-4771-b299-0c05c598a109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:41.856721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586d367-3c15-4771-b299-0c05c598a109 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:41.857080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586d367-3c15-4771-b299-0c05c598a109 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:41.857433 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586d367-3c15-4771-b299-0c05c598a109 admin admin] PATH_INFO: `/v3/projects/8c3acf54deb947988ed5109422c9ce2f/users/7d99853ee2d2418c9b1714eb5321c0d9/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:41.860523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc55a4f-35e7-4988-90cb-44abe413e765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:41.861170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc55a4f-35e7-4988-90cb-44abe413e765 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:41.861450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc55a4f-35e7-4988-90cb-44abe413e765 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:41.861725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc55a4f-35e7-4988-90cb-44abe413e765 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:41.862468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc55a4f-35e7-4988-90cb-44abe413e765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:41.864421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc55a4f-35e7-4988-90cb-44abe413e765 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:41.878419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586d367-3c15-4771-b299-0c05c598a109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c3acf54deb947988ed5109422c9ce2f, user_id=7d99853ee2d2418c9b1714eb5321c0d9, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:41.882990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586d367-3c15-4771-b299-0c05c598a109 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:41.996931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 883/1764] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:41 2022] PUT /identity/v3/projects/8c3acf54deb947988ed5109422c9ce2f/users/7d99853ee2d2418c9b1714eb5321c0d9/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:42.001060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586d367-3c15-4771-b299-0c05c598a109 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:42.003153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 882/1765] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:41 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:42.007320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc55a4f-35e7-4988-90cb-44abe413e765 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:42.189123 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc55a4f-35e7-4988-90cb-44abe413e765 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:42.214544 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97436e0a-d384-46cb-b99a-282514d466e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:42.215067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586d367-3c15-4771-b299-0c05c598a109 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:42.215167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97436e0a-d384-46cb-b99a-282514d466e7 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:42.215167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97436e0a-d384-46cb-b99a-282514d466e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:42.215167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97436e0a-d384-46cb-b99a-282514d466e7 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:42.215167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97436e0a-d384-46cb-b99a-282514d466e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:42.215167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97436e0a-d384-46cb-b99a-282514d466e7 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:42.215613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c9b82-c271-43de-b32d-6ed07fd00429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:42.215613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c9b82-c271-43de-b32d-6ed07fd00429 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:42.215613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c9b82-c271-43de-b32d-6ed07fd00429 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:42.215613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c9b82-c271-43de-b32d-6ed07fd00429 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:42.215922 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c9b82-c271-43de-b32d-6ed07fd00429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:42.215922 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c9b82-c271-43de-b32d-6ed07fd00429 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:42.229932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 884/1766] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:42.231059 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c9b82-c271-43de-b32d-6ed07fd00429 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:42.242444 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c9b82-c271-43de-b32d-6ed07fd00429 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:42.249120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe8e24-ca0a-4e0b-9793-018ba8f4cbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:42.249732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe8e24-ca0a-4e0b-9793-018ba8f4cbd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:42.250043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe8e24-ca0a-4e0b-9793-018ba8f4cbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:42.250345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe8e24-ca0a-4e0b-9793-018ba8f4cbd7 admin admin] PATH_INFO: `/v3/projects/8c3acf54deb947988ed5109422c9ce2f/users/7d99853ee2d2418c9b1714eb5321c0d9/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:42.252346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe8e24-ca0a-4e0b-9793-018ba8f4cbd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c3acf54deb947988ed5109422c9ce2f, user_id=7d99853ee2d2418c9b1714eb5321c0d9, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:42.255218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe8e24-ca0a-4e0b-9793-018ba8f4cbd7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:42.267842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 885/1767] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:42 2022] PUT /identity/v3/projects/8c3acf54deb947988ed5109422c9ce2f/users/7d99853ee2d2418c9b1714eb5321c0d9/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:42.279538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eb6ba-9244-43b1-9862-93df12d2b776 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:42.280146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eb6ba-9244-43b1-9862-93df12d2b776 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:42.280555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eb6ba-9244-43b1-9862-93df12d2b776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:42.328095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-4a4eb6ba-9244-43b1-9862-93df12d2b776 None None] MFA Rules not processed for user `7d99853ee2d2418c9b1714eb5321c0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:42.329455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4a4eb6ba-9244-43b1-9862-93df12d2b776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:42.382575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 883/1768] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:42 2022] POST /identity/v3/users => generated 294 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:42.386941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97436e0a-d384-46cb-b99a-282514d466e7 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:42.411119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97436e0a-d384-46cb-b99a-282514d466e7 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:42.457789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 886/1769] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:42.494294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4eb6ba-9244-43b1-9862-93df12d2b776 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:42.497797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4a4eb6ba-9244-43b1-9862-93df12d2b776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:42.634575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f55291-fbbe-4a44-8e18-5f35575358ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:42.636008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f55291-fbbe-4a44-8e18-5f35575358ab admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:42.636435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f55291-fbbe-4a44-8e18-5f35575358ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:42.636812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f55291-fbbe-4a44-8e18-5f35575358ab admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:42.637825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f55291-fbbe-4a44-8e18-5f35575358ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:42.638401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4eb6ba-9244-43b1-9862-93df12d2b776 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:42.639726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f55291-fbbe-4a44-8e18-5f35575358ab admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:42.643792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0678cb-77a2-44e3-a3a1-4d5b8b344c80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:42.643985 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0678cb-77a2-44e3-a3a1-4d5b8b344c80 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:42.644567 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0678cb-77a2-44e3-a3a1-4d5b8b344c80 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:42.644693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0678cb-77a2-44e3-a3a1-4d5b8b344c80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:42.655619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 884/1770] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:40:42.655619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87eef5-5f22-4726-8bbb-44b5d5f0e1d8 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:42.655619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87eef5-5f22-4726-8bbb-44b5d5f0e1d8 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:42.655619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87eef5-5f22-4726-8bbb-44b5d5f0e1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:42.659153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0678cb-77a2-44e3-a3a1-4d5b8b344c80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:42.660594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0678cb-77a2-44e3-a3a1-4d5b8b344c80 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:42.680337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 887/1771] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:40:42.684544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0678cb-77a2-44e3-a3a1-4d5b8b344c80 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:42.699199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0678cb-77a2-44e3-a3a1-4d5b8b344c80 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:42.715316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea799cde-ecfd-4768-a858-5a51f094c548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:42.718147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799cde-ecfd-4768-a858-5a51f094c548 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:42.718147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799cde-ecfd-4768-a858-5a51f094c548 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:42.718147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799cde-ecfd-4768-a858-5a51f094c548 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:42.718147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea799cde-ecfd-4768-a858-5a51f094c548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:42.718834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea799cde-ecfd-4768-a858-5a51f094c548 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:42.745236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 888/1772] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:42.749609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea799cde-ecfd-4768-a858-5a51f094c548 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:42.754262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-8a87eef5-5f22-4726-8bbb-44b5d5f0e1d8 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:42.755563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8a87eef5-5f22-4726-8bbb-44b5d5f0e1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:42.765895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea799cde-ecfd-4768-a858-5a51f094c548 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:42.773033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0d520-f669-402b-8c72-50825ecb9415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:42.774109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d520-f669-402b-8c72-50825ecb9415 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:42.774399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d520-f669-402b-8c72-50825ecb9415 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:42.774765 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d520-f669-402b-8c72-50825ecb9415 admin admin] PATH_INFO: `/v3/projects/107ae0b8fa2d4f81a0be5f562c096358/users/b95c18aa11d048a185d0f1781a9538f1/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:42.798378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0d520-f669-402b-8c72-50825ecb9415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=107ae0b8fa2d4f81a0be5f562c096358, user_id=b95c18aa11d048a185d0f1781a9538f1, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:42.801634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0d520-f669-402b-8c72-50825ecb9415 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:42.893015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 889/1773] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:42 2022] PUT /identity/v3/projects/107ae0b8fa2d4f81a0be5f562c096358/users/b95c18aa11d048a185d0f1781a9538f1/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:42.908407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a8240-4e26-4f1c-b0fc-c45999dc81ee None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:42.908699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a8240-4e26-4f1c-b0fc-c45999dc81ee None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:42.908974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a8240-4e26-4f1c-b0fc-c45999dc81ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:42.935458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 885/1774] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:40:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:42.974603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ea2a8240-4e26-4f1c-b0fc-c45999dc81ee None None] MFA Rules not processed for user `b95c18aa11d048a185d0f1781a9538f1`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:42.977864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ea2a8240-4e26-4f1c-b0fc-c45999dc81ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:43.067426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 890/1775] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:43.072090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87eef5-5f22-4726-8bbb-44b5d5f0e1d8 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:43.105579 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87eef5-5f22-4726-8bbb-44b5d5f0e1d8 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:43.236679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405c0cf-3de0-43ea-8b9d-e20c6ee9a0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:43.237617 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405c0cf-3de0-43ea-8b9d-e20c6ee9a0bc admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:43.237913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405c0cf-3de0-43ea-8b9d-e20c6ee9a0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:43.238324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405c0cf-3de0-43ea-8b9d-e20c6ee9a0bc admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:43.239878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405c0cf-3de0-43ea-8b9d-e20c6ee9a0bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:43.242437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405c0cf-3de0-43ea-8b9d-e20c6ee9a0bc admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:43.420486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a8240-4e26-4f1c-b0fc-c45999dc81ee None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:43.422085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ea2a8240-4e26-4f1c-b0fc-c45999dc81ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:43.509886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 886/1776] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:43 2022] POST /identity/v3/projects => generated 385 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:43.516557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405c0cf-3de0-43ea-8b9d-e20c6ee9a0bc admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:43.712461 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405c0cf-3de0-43ea-8b9d-e20c6ee9a0bc admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:43.717668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a8240-4e26-4f1c-b0fc-c45999dc81ee None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:43.738188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac1a0e-3d43-4149-9931-bd8726b29c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:43.738990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac1a0e-3d43-4149-9931-bd8726b29c2d admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:43.739276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac1a0e-3d43-4149-9931-bd8726b29c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:43.739565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac1a0e-3d43-4149-9931-bd8726b29c2d admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:43.742501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac1a0e-3d43-4149-9931-bd8726b29c2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:43.748569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac1a0e-3d43-4149-9931-bd8726b29c2d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:43.867267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 887/1777] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:43 2022] POST /identity/v3/users => generated 299 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:43.871337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481ec48-1325-4aa5-a0bf-d22675e947f7 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:43.871337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481ec48-1325-4aa5-a0bf-d22675e947f7 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:43.871337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481ec48-1325-4aa5-a0bf-d22675e947f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:43.874573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737195e8-7c7d-4c7a-ad5e-a9b65d728f87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:43.875200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737195e8-7c7d-4c7a-ad5e-a9b65d728f87 service swift] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:43.875488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737195e8-7c7d-4c7a-ad5e-a9b65d728f87 service swift] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:43.875765 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737195e8-7c7d-4c7a-ad5e-a9b65d728f87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:43.919020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c481ec48-1325-4aa5-a0bf-d22675e947f7 None None] MFA Rules not processed for user `c98761f81496456fb3b5a5cffd4ce763`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:43.924296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c481ec48-1325-4aa5-a0bf-d22675e947f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:44.054421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 888/1778] 10.222.0.95 () {58 vars in 1015 bytes} [Thu Aug 4 21:40:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:44.055308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481ec48-1325-4aa5-a0bf-d22675e947f7 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:44.068131 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737195e8-7c7d-4c7a-ad5e-a9b65d728f87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:44.075294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481ec48-1325-4aa5-a0bf-d22675e947f7 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:44.087129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e265a39-aee5-4208-acd2-c2f8f7905869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:44.087129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e265a39-aee5-4208-acd2-c2f8f7905869 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:44.087129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e265a39-aee5-4208-acd2-c2f8f7905869 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:44.087129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e265a39-aee5-4208-acd2-c2f8f7905869 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:44.087433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e265a39-aee5-4208-acd2-c2f8f7905869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:44.090404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e265a39-aee5-4208-acd2-c2f8f7905869 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:44.095678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737195e8-7c7d-4c7a-ad5e-a9b65d728f87 service swift] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:44.107955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 889/1779] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:44.109138 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e265a39-aee5-4208-acd2-c2f8f7905869 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:44.111064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 891/1780] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:40:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:40:44.132453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737195e8-7c7d-4c7a-ad5e-a9b65d728f87 service swift] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:44.137113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-737195e8-7c7d-4c7a-ad5e-a9b65d728f87 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:44.139578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e265a39-aee5-4208-acd2-c2f8f7905869 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:44.154768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858badea-9699-42ac-8d9c-e4a634be9c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:44.160670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737195e8-7c7d-4c7a-ad5e-a9b65d728f87 service swift] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:44.160794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858badea-9699-42ac-8d9c-e4a634be9c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:44.160794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858badea-9699-42ac-8d9c-e4a634be9c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:44.160794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858badea-9699-42ac-8d9c-e4a634be9c5c admin admin] PATH_INFO: `/v3/projects/c5f346e3ae5743769c367f31107fd578/users/e61db0614882492291b23227461d298c/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:44.163334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfd752-4cb4-4ff7-8363-73b80efc81d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:44.164241 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfd752-4cb4-4ff7-8363-73b80efc81d7 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:44.164694 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfd752-4cb4-4ff7-8363-73b80efc81d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:44.165965 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfd752-4cb4-4ff7-8363-73b80efc81d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:44.176933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfd752-4cb4-4ff7-8363-73b80efc81d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:44.180230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfd752-4cb4-4ff7-8363-73b80efc81d7 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:44.193261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 892/1781] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:44.209015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858badea-9699-42ac-8d9c-e4a634be9c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f346e3ae5743769c367f31107fd578, user_id=e61db0614882492291b23227461d298c, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:44.217179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858badea-9699-42ac-8d9c-e4a634be9c5c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:44.368659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 890/1782] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:44 2022] PUT /identity/v3/projects/c5f346e3ae5743769c367f31107fd578/users/e61db0614882492291b23227461d298c/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:44.373897 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfd752-4cb4-4ff7-8363-73b80efc81d7 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:44.388029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:44.406348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfd752-4cb4-4ff7-8363-73b80efc81d7 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:44.559658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:44.570959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff13b3-9e47-4024-9dbb-c9d1b6b10348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:44.571618 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff13b3-9e47-4024-9dbb-c9d1b6b10348 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:44.571905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff13b3-9e47-4024-9dbb-c9d1b6b10348 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:44.572235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff13b3-9e47-4024-9dbb-c9d1b6b10348 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:44.575804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff13b3-9e47-4024-9dbb-c9d1b6b10348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:44.580530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff13b3-9e47-4024-9dbb-c9d1b6b10348 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:44.619030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858badea-9699-42ac-8d9c-e4a634be9c5c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:44.633995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858badea-9699-42ac-8d9c-e4a634be9c5c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:44.639064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 893/1783] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:44.653089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a114771-7784-4641-9fd5-e6397687852e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:44.654540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a114771-7784-4641-9fd5-e6397687852e admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:44.654833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a114771-7784-4641-9fd5-e6397687852e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:44.655115 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a114771-7784-4641-9fd5-e6397687852e admin admin] PATH_INFO: `/v3/projects/c5f346e3ae5743769c367f31107fd578/users/e61db0614882492291b23227461d298c/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:44.662078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a114771-7784-4641-9fd5-e6397687852e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f346e3ae5743769c367f31107fd578, user_id=e61db0614882492291b23227461d298c, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:44.669649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a114771-7784-4641-9fd5-e6397687852e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:44.739009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7eefa-3831-40d4-b6e1-af54e1da8ab4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:44.739301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7eefa-3831-40d4-b6e1-af54e1da8ab4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:44.744177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7eefa-3831-40d4-b6e1-af54e1da8ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:44.744340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 891/1784] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:44 2022] PUT /identity/v3/projects/c5f346e3ae5743769c367f31107fd578/users/e61db0614882492291b23227461d298c/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:44.848300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-42f7eefa-3831-40d4-b6e1-af54e1da8ab4 None None] MFA Rules not processed for user `e61db0614882492291b23227461d298c`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:44.849574 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-42f7eefa-3831-40d4-b6e1-af54e1da8ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:45.176469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 894/1785] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:45.299769 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a114771-7784-4641-9fd5-e6397687852e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:45.302134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4a114771-7784-4641-9fd5-e6397687852e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:45.506363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a114771-7784-4641-9fd5-e6397687852e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:45.511252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec227e81-aded-4dc9-9ccc-68c92b0966a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:45.511885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec227e81-aded-4dc9-9ccc-68c92b0966a7 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:45.512366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec227e81-aded-4dc9-9ccc-68c92b0966a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:45.513450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec227e81-aded-4dc9-9ccc-68c92b0966a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:45.545783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec227e81-aded-4dc9-9ccc-68c92b0966a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:45.547906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec227e81-aded-4dc9-9ccc-68c92b0966a7 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:45.564632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 892/1786] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:45.854504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7eefa-3831-40d4-b6e1-af54e1da8ab4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:46.131256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7eefa-3831-40d4-b6e1-af54e1da8ab4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:46.152357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:46.153029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:46.153310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:46.153581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] PATH_INFO: `/v3/users/b95c18aa11d048a185d0f1781a9538f1` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:46.165003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b95c18aa11d048a185d0f1781a9538f1)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:46.170340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:46.513408 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b95c18aa11d048a185d0f1781a9538f1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:46.601968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:46.658635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b95c18aa11d048a185d0f1781a9538f1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:46.685711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b95c18aa11d048a185d0f1781a9538f1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:46.691889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b95c18aa11d048a185d0f1781a9538f1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:46.712380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b95c18aa11d048a185d0f1781a9538f1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:46.721688 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 895/1787] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:45 2022] DELETE /identity/v3/users/b95c18aa11d048a185d0f1781a9538f1 => generated 0 bytes in 869 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:46.727294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebec777-cdc5-4ea2-8fac-c90e60ffcd11 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:46.727294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebec777-cdc5-4ea2-8fac-c90e60ffcd11 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:46.727294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebec777-cdc5-4ea2-8fac-c90e60ffcd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:46.748596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:46.778162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-4ebec777-cdc5-4ea2-8fac-c90e60ffcd11 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:46.808099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4ebec777-cdc5-4ea2-8fac-c90e60ffcd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:46.952641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 893/1788] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:46.969078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:46.975726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:47.113768 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428d5cf-c21a-4782-949c-fc2f7f123012 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:47.118907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f01ab-9a3c-4064-86da-e920c9ba28fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:47.119814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f01ab-9a3c-4064-86da-e920c9ba28fc service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:47.120165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f01ab-9a3c-4064-86da-e920c9ba28fc service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:47.120516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f01ab-9a3c-4064-86da-e920c9ba28fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:47.134475 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f01ab-9a3c-4064-86da-e920c9ba28fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:47.136032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f01ab-9a3c-4064-86da-e920c9ba28fc service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:47.146282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 896/1789] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:47.611285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=78,keystone:DELETE=17,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:47.753069 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5c156-75e1-4c5b-908c-0aa1310f2e00 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:47.753442 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5c156-75e1-4c5b-908c-0aa1310f2e00 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:47.753962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5c156-75e1-4c5b-908c-0aa1310f2e00 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:47.755746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 894/1790] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:40:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:40:47.766465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1477839-59e1-4e06-a15a-cb5dc907fe09 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:47.766465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1477839-59e1-4e06-a15a-cb5dc907fe09 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:47.767136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1477839-59e1-4e06-a15a-cb5dc907fe09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:47.805466 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-b1477839-59e1-4e06-a15a-cb5dc907fe09 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:47.807590 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b1477839-59e1-4e06-a15a-cb5dc907fe09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:47.909531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 897/1791] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:40:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:47.928009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5c156-75e1-4c5b-908c-0aa1310f2e00 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:47.930405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0de5c156-75e1-4c5b-908c-0aa1310f2e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:47.960041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5c156-75e1-4c5b-908c-0aa1310f2e00 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:47.968385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2e7b0-dadc-4cee-8657-f56d6173048a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:47.969090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2e7b0-dadc-4cee-8657-f56d6173048a service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:47.969671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2e7b0-dadc-4cee-8657-f56d6173048a service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:47.970087 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2e7b0-dadc-4cee-8657-f56d6173048a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:47.985675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2e7b0-dadc-4cee-8657-f56d6173048a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:47.987871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2e7b0-dadc-4cee-8657-f56d6173048a service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:48.016505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 895/1792] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:48.124038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1477839-59e1-4e06-a15a-cb5dc907fe09 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:48.125573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b1477839-59e1-4e06-a15a-cb5dc907fe09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:48.150755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1477839-59e1-4e06-a15a-cb5dc907fe09 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:48.158699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:48.159370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:48.159656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:48.159987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] PATH_INFO: `/v3/projects/107ae0b8fa2d4f81a0be5f562c096358` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:48.163038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=107ae0b8fa2d4f81a0be5f562c096358)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:48.166711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:48.239068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa3c74-dd57-4f48-9c4c-89ec3c64ffb0 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:48.239357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa3c74-dd57-4f48-9c4c-89ec3c64ffb0 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:48.241840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa3c74-dd57-4f48-9c4c-89ec3c64ffb0 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:48.245257 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 896/1793] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:40:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:40:48.301425 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e200d6b-d2e9-4798-8bc8-a1367b575344 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:48.303342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e200d6b-d2e9-4798-8bc8-a1367b575344 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:48.307749 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e200d6b-d2e9-4798-8bc8-a1367b575344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:48.336729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-3e200d6b-d2e9-4798-8bc8-a1367b575344 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:48.339805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3e200d6b-d2e9-4798-8bc8-a1367b575344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:48.342747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '107ae0b8fa2d4f81a0be5f562c096358', 'request_id': 'req-ab88c55c-b3fc-456b-8788-ac6754f4dc92'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:48.431360 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 897/1794] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:40:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:48.445333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e200d6b-d2e9-4798-8bc8-a1367b575344 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:48.446280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '107ae0b8fa2d4f81a0be5f562c096358', 'request_id': 'req-ab88c55c-b3fc-456b-8788-ac6754f4dc92'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:48.447341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3e200d6b-d2e9-4798-8bc8-a1367b575344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:48.454544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '107ae0b8fa2d4f81a0be5f562c096358', 'request_id': 'req-ab88c55c-b3fc-456b-8788-ac6754f4dc92'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:48.467176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e200d6b-d2e9-4798-8bc8-a1367b575344 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:48.471863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] The token cache is being invalidate because project 107ae0b8fa2d4f81a0be5f562c096358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:48.472416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:48.476426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:48.476426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 898/1795] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:48 2022] DELETE /identity/v3/projects/107ae0b8fa2d4f81a0be5f562c096358 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:48.479741 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:48.481014 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a870d6-db80-47ed-97c5-f58928068fd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:48.481285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:48.482194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a870d6-db80-47ed-97c5-f58928068fd7 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:48.482734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a870d6-db80-47ed-97c5-f58928068fd7 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:48.483228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a870d6-db80-47ed-97c5-f58928068fd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:48.487556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-17a870d6-db80-47ed-97c5-f58928068fd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:48.513235 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a870d6-db80-47ed-97c5-f58928068fd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:48.515134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a870d6-db80-47ed-97c5-f58928068fd7 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:48.528316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 898/1796] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:48.571261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88c55c-b3fc-456b-8788-ac6754f4dc92 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:48.578153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d64636-ec9a-4f2d-b6e8-73230c107ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:48.579411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d64636-ec9a-4f2d-b6e8-73230c107ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:48.580584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d64636-ec9a-4f2d-b6e8-73230c107ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:48.581048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d64636-ec9a-4f2d-b6e8-73230c107ee2 admin admin] PATH_INFO: `/v3/users/e61db0614882492291b23227461d298c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:48.583137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d64636-ec9a-4f2d-b6e8-73230c107ee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61db0614882492291b23227461d298c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:48.586365 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d64636-ec9a-4f2d-b6e8-73230c107ee2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:48.733823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:48.823938 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-97d64636-ec9a-4f2d-b6e8-73230c107ee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61db0614882492291b23227461d298c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:48.906484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-97d64636-ec9a-4f2d-b6e8-73230c107ee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61db0614882492291b23227461d298c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:48.920392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a870d6-db80-47ed-97c5-f58928068fd7 service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:48.922414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-97d64636-ec9a-4f2d-b6e8-73230c107ee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61db0614882492291b23227461d298c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:48.922414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-97d64636-ec9a-4f2d-b6e8-73230c107ee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61db0614882492291b23227461d298c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:48.923167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-17a870d6-db80-47ed-97c5-f58928068fd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:48.933720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-97d64636-ec9a-4f2d-b6e8-73230c107ee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61db0614882492291b23227461d298c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:48.936874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a870d6-db80-47ed-97c5-f58928068fd7 service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:48.939968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 899/1797] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:48 2022] DELETE /identity/v3/users/e61db0614882492291b23227461d298c => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:48.946603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:48.950984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:48.950984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:48.950984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] PATH_INFO: `/v3/users/e7bf83d042634f6abd5968c9b36de102` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:48.950984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7bf83d042634f6abd5968c9b36de102)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:48.952245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:49.156864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7bf83d042634f6abd5968c9b36de102'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:49.233102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7bf83d042634f6abd5968c9b36de102'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:49.234043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7bf83d042634f6abd5968c9b36de102'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:49.251252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7bf83d042634f6abd5968c9b36de102'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:49.259227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7bf83d042634f6abd5968c9b36de102'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:49.267411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 899/1798] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:48 2022] DELETE /identity/v3/users/e7bf83d042634f6abd5968c9b36de102 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:49.272024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187436e-0846-4827-af7a-186b32abf852 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:49.273801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187436e-0846-4827-af7a-186b32abf852 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:49.274115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187436e-0846-4827-af7a-186b32abf852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:49.299310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-7187436e-0846-4827-af7a-186b32abf852 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:49.300557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7187436e-0846-4827-af7a-186b32abf852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:49.363139 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 900/1799] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:49.373314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:49.374749 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:49.390064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187436e-0846-4827-af7a-186b32abf852 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:49.391956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7187436e-0846-4827-af7a-186b32abf852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:49.405228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187436e-0846-4827-af7a-186b32abf852 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:49.432249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:49.432249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:49.432249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:49.432249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] PATH_INFO: `/v3/projects/c5f346e3ae5743769c367f31107fd578` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:49.434148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f346e3ae5743769c367f31107fd578)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:49.436720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:49.467458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab0d10-b442-473e-a081-59d23481dec2 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:49.487214 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a0059-1d4f-4ed3-a946-591f91e5b494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:49.488129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a0059-1d4f-4ed3-a946-591f91e5b494 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:49.488578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a0059-1d4f-4ed3-a946-591f91e5b494 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:49.488850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a0059-1d4f-4ed3-a946-591f91e5b494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:49.503099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a0059-1d4f-4ed3-a946-591f91e5b494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:49.504632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a0059-1d4f-4ed3-a946-591f91e5b494 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:49.519292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 900/1800] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:49.614571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f346e3ae5743769c367f31107fd578', 'request_id': 'req-1de30f63-c74b-4f2c-9d91-33392bf32ecf'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:49.777524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f346e3ae5743769c367f31107fd578', 'request_id': 'req-1de30f63-c74b-4f2c-9d91-33392bf32ecf'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:49.788902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f346e3ae5743769c367f31107fd578', 'request_id': 'req-1de30f63-c74b-4f2c-9d91-33392bf32ecf'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:49.803989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] The token cache is being invalidate because project c5f346e3ae5743769c367f31107fd578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:49.804454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:49.807619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1de30f63-c74b-4f2c-9d91-33392bf32ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:49.813568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 901/1801] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:49 2022] DELETE /identity/v3/projects/c5f346e3ae5743769c367f31107fd578 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:49.821461 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e0e32-7cb1-4d08-b328-aa6b84572da2 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:49.821840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e0e32-7cb1-4d08-b328-aa6b84572da2 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:49.822155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e0e32-7cb1-4d08-b328-aa6b84572da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:49.854282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-db2e0e32-7cb1-4d08-b328-aa6b84572da2 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:49.855573 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-db2e0e32-7cb1-4d08-b328-aa6b84572da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:49.962524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 901/1802] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:49.965633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1780d50-4bde-4f44-b096-a99f5d44cbd7 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:49.974146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1780d50-4bde-4f44-b096-a99f5d44cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:49.974146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1780d50-4bde-4f44-b096-a99f5d44cbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:50.001568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-d1780d50-4bde-4f44-b096-a99f5d44cbd7 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:50.005189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d1780d50-4bde-4f44-b096-a99f5d44cbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:50.038249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 902/1803] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:50.038730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e0e32-7cb1-4d08-b328-aa6b84572da2 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:50.059217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e0e32-7cb1-4d08-b328-aa6b84572da2 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:50.064601 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d233780-c358-487f-8027-40907adb2eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:50.070118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d233780-c358-487f-8027-40907adb2eba admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:50.070118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d233780-c358-487f-8027-40907adb2eba admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:50.070118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d233780-c358-487f-8027-40907adb2eba admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:50.070118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d233780-c358-487f-8027-40907adb2eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:50.070118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d233780-c358-487f-8027-40907adb2eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:50.070118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d233780-c358-487f-8027-40907adb2eba admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:50.083147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 902/1804] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:50.088938 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1780d50-4bde-4f44-b096-a99f5d44cbd7 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:50.100690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1780d50-4bde-4f44-b096-a99f5d44cbd7 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:50.113611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49977feb-4c28-4078-a513-981869a47d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:50.114318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49977feb-4c28-4078-a513-981869a47d57 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:50.114745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49977feb-4c28-4078-a513-981869a47d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:50.115028 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49977feb-4c28-4078-a513-981869a47d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:50.115707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49977feb-4c28-4078-a513-981869a47d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:50.117843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49977feb-4c28-4078-a513-981869a47d57 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:50.124705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:50.277512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d233780-c358-487f-8027-40907adb2eba admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:50.279084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 903/1805] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:50 2022] POST /identity/v3/projects => generated 381 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:50.280473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8d233780-c358-487f-8027-40907adb2eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:50.302294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49977feb-4c28-4078-a513-981869a47d57 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:50.315512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d233780-c358-487f-8027-40907adb2eba admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:50.382928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49977feb-4c28-4078-a513-981869a47d57 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:50.388531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251429a1-7a12-473d-a140-f11e261ebfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:50.389457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251429a1-7a12-473d-a140-f11e261ebfc2 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:50.389922 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251429a1-7a12-473d-a140-f11e261ebfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:50.390483 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251429a1-7a12-473d-a140-f11e261ebfc2 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:50.390770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:50.391757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251429a1-7a12-473d-a140-f11e261ebfc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:50.393433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:50.393790 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:50.394095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] PATH_INFO: `/v3/projects/2e86dc866a794e1288aa71b83556693f` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:50.394834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251429a1-7a12-473d-a140-f11e261ebfc2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:50.398766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e86dc866a794e1288aa71b83556693f)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:50.398766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:50.476042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 904/1806] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:50 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:50.480492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251429a1-7a12-473d-a140-f11e261ebfc2 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:50.492269 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251429a1-7a12-473d-a140-f11e261ebfc2 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:50.498748 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069336ec-a272-4062-bc2e-7ad8db9845b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:50.499373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069336ec-a272-4062-bc2e-7ad8db9845b9 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:50.499659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069336ec-a272-4062-bc2e-7ad8db9845b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:50.499933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069336ec-a272-4062-bc2e-7ad8db9845b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:50.500644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069336ec-a272-4062-bc2e-7ad8db9845b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:50.502309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069336ec-a272-4062-bc2e-7ad8db9845b9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:50.517888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 905/1807] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:50.520804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069336ec-a272-4062-bc2e-7ad8db9845b9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:50.535995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e86dc866a794e1288aa71b83556693f', 'request_id': 'req-0c2323f3-0072-4d30-9e24-f2646412c5bd'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:50.536630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069336ec-a272-4062-bc2e-7ad8db9845b9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:50.545000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e86dc866a794e1288aa71b83556693f', 'request_id': 'req-0c2323f3-0072-4d30-9e24-f2646412c5bd'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:50.604210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e86dc866a794e1288aa71b83556693f', 'request_id': 'req-0c2323f3-0072-4d30-9e24-f2646412c5bd'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:50.608676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913c013-1715-442d-82f7-be07739d9c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:50.609384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913c013-1715-442d-82f7-be07739d9c49 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:50.609754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913c013-1715-442d-82f7-be07739d9c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:50.610029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913c013-1715-442d-82f7-be07739d9c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:50.611077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913c013-1715-442d-82f7-be07739d9c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:50.613178 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913c013-1715-442d-82f7-be07739d9c49 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:50.625857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] The token cache is being invalidate because project 2e86dc866a794e1288aa71b83556693f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:50.626360 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:50.627190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:50.634266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 906/1808] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:40:50.635756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 903/1809] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:50 2022] DELETE /identity/v3/projects/2e86dc866a794e1288aa71b83556693f => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:50.640465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:50.641148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913c013-1715-442d-82f7-be07739d9c49 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:50.642393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:50.643069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f913c013-1715-442d-82f7-be07739d9c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:50.657187 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2323f3-0072-4d30-9e24-f2646412c5bd admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:50.657757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913c013-1715-442d-82f7-be07739d9c49 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:50.665221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf173e4-5258-44bf-9dc1-d86dae2b1ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:50.667055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf173e4-5258-44bf-9dc1-d86dae2b1ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:50.667380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf173e4-5258-44bf-9dc1-d86dae2b1ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:50.667738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf173e4-5258-44bf-9dc1-d86dae2b1ba0 admin admin] PATH_INFO: `/v3/projects/c45276ef762c4b77bc7953a6bd74d983/users/c351fe4452ee4561a6c087f40b8b2366/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:50.683805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:50.684196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:50.684484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:50.684767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] PATH_INFO: `/v3/users/7d99853ee2d2418c9b1714eb5321c0d9` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:50.687886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d99853ee2d2418c9b1714eb5321c0d9)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:50.689924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:50.697522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf173e4-5258-44bf-9dc1-d86dae2b1ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45276ef762c4b77bc7953a6bd74d983, user_id=c351fe4452ee4561a6c087f40b8b2366, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:50.702410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf173e4-5258-44bf-9dc1-d86dae2b1ba0 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:50.821464 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 904/1810] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:50 2022] PUT /identity/v3/projects/c45276ef762c4b77bc7953a6bd74d983/users/c351fe4452ee4561a6c087f40b8b2366/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:50.830730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390b397-fed3-4919-8b67-0b20e15a9678 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:50.831055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390b397-fed3-4919-8b67-0b20e15a9678 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:50.831331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390b397-fed3-4919-8b67-0b20e15a9678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:50.870672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-7390b397-fed3-4919-8b67-0b20e15a9678 None None] MFA Rules not processed for user `c351fe4452ee4561a6c087f40b8b2366`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:50.872539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-7390b397-fed3-4919-8b67-0b20e15a9678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:50.937156 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d99853ee2d2418c9b1714eb5321c0d9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:50.948340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 905/1811] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:50.952921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390b397-fed3-4919-8b67-0b20e15a9678 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:50.966782 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:50.989664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d99853ee2d2418c9b1714eb5321c0d9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:51.004342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d99853ee2d2418c9b1714eb5321c0d9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:51.005795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d99853ee2d2418c9b1714eb5321c0d9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:51.029459 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d99853ee2d2418c9b1714eb5321c0d9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:51.036695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 907/1812] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:50 2022] DELETE /identity/v3/users/7d99853ee2d2418c9b1714eb5321c0d9 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:51.044924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390b397-fed3-4919-8b67-0b20e15a9678 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:51.185629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cbb18-75bb-43d1-8309-0a456f1cd426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:51.188521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cbb18-75bb-43d1-8309-0a456f1cd426 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:51.202603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cbb18-75bb-43d1-8309-0a456f1cd426 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:51.202603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cbb18-75bb-43d1-8309-0a456f1cd426 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:51.202603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cbb18-75bb-43d1-8309-0a456f1cd426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:51.202603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cbb18-75bb-43d1-8309-0a456f1cd426 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:51.348190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 906/1813] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:50 2022] POST /identity/v3/projects => generated 379 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:51.349345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:51.350837 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:51.368808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a150e3-b371-4de8-be5b-3a984ab9b9ef admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:51.484643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080098b-a087-47c3-abee-aebef1e041e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:51.488195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080098b-a087-47c3-abee-aebef1e041e3 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:51.489330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080098b-a087-47c3-abee-aebef1e041e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:51.490540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080098b-a087-47c3-abee-aebef1e041e3 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:51.495263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080098b-a087-47c3-abee-aebef1e041e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:51.495710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:51.495865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080098b-a087-47c3-abee-aebef1e041e3 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:51.603704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 908/1814] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:51 2022] POST /identity/v3/users => generated 296 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:51.605250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cbb18-75bb-43d1-8309-0a456f1cd426 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:51.619908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cbb18-75bb-43d1-8309-0a456f1cd426 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:51.624834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c70c7-c48f-44d5-b400-b547a3e41baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:51.625524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c70c7-c48f-44d5-b400-b547a3e41baf admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:51.625873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c70c7-c48f-44d5-b400-b547a3e41baf admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:51.626857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c70c7-c48f-44d5-b400-b547a3e41baf admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:51.626987 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c70c7-c48f-44d5-b400-b547a3e41baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:51.628766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c70c7-c48f-44d5-b400-b547a3e41baf admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:51.640382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 907/1815] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:40:51.647533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080098b-a087-47c3-abee-aebef1e041e3 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:51.658093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080098b-a087-47c3-abee-aebef1e041e3 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:51.664128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f0512-dd2b-4fbc-9382-37359e359b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:51.664801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0512-dd2b-4fbc-9382-37359e359b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:51.665095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0512-dd2b-4fbc-9382-37359e359b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:51.665405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0512-dd2b-4fbc-9382-37359e359b9d admin admin] PATH_INFO: `/v3/projects/12d99dc2e48e41179cbe98a2e3c69d2d/users/2149f4b86e064e2e98575720ae43e109/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:51.684790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f0512-dd2b-4fbc-9382-37359e359b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=12d99dc2e48e41179cbe98a2e3c69d2d, user_id=2149f4b86e064e2e98575720ae43e109, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:51.687470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f0512-dd2b-4fbc-9382-37359e359b9d admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:51.727707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c70c7-c48f-44d5-b400-b547a3e41baf admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:51.729741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-de4c70c7-c48f-44d5-b400-b547a3e41baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:51.743508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c70c7-c48f-44d5-b400-b547a3e41baf admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:51.750561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:51.751220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:51.757588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:51.757588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] PATH_INFO: `/v3/projects/8c3acf54deb947988ed5109422c9ce2f` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:51.757588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c3acf54deb947988ed5109422c9ce2f)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:51.757588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:51.791954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 909/1816] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:51 2022] PUT /identity/v3/projects/12d99dc2e48e41179cbe98a2e3c69d2d/users/2149f4b86e064e2e98575720ae43e109/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:51.796283 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f0512-dd2b-4fbc-9382-37359e359b9d admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:51.861855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f0512-dd2b-4fbc-9382-37359e359b9d admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:51.866879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85c608-9ee6-4db0-b320-469476e1785b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:51.867503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85c608-9ee6-4db0-b320-469476e1785b admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:51.867788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85c608-9ee6-4db0-b320-469476e1785b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:51.868072 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85c608-9ee6-4db0-b320-469476e1785b admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:51.868767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85c608-9ee6-4db0-b320-469476e1785b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:51.871336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85c608-9ee6-4db0-b320-469476e1785b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:51.881450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 910/1817] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:51.886533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85c608-9ee6-4db0-b320-469476e1785b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:51.896492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85c608-9ee6-4db0-b320-469476e1785b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:51.905173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c3acf54deb947988ed5109422c9ce2f', 'request_id': 'req-76c33440-9872-4bb9-b1d7-fd610c8a1082'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:51.913711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c3acf54deb947988ed5109422c9ce2f', 'request_id': 'req-76c33440-9872-4bb9-b1d7-fd610c8a1082'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:51.965093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f3575-3466-42ee-94bb-7a720089ad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:51.965794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3575-3466-42ee-94bb-7a720089ad47 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:51.966106 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3575-3466-42ee-94bb-7a720089ad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:51.972754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3575-3466-42ee-94bb-7a720089ad47 admin admin] PATH_INFO: `/v3/projects/12d99dc2e48e41179cbe98a2e3c69d2d/users/2149f4b86e064e2e98575720ae43e109/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:51.972754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f3575-3466-42ee-94bb-7a720089ad47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12d99dc2e48e41179cbe98a2e3c69d2d, user_id=2149f4b86e064e2e98575720ae43e109, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:51.972754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f3575-3466-42ee-94bb-7a720089ad47 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:51.982680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 911/1818] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:51 2022] PUT /identity/v3/projects/12d99dc2e48e41179cbe98a2e3c69d2d/users/2149f4b86e064e2e98575720ae43e109/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:51.995098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a57b6a-846d-4a4e-a590-9658be0622e6 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:51.995392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a57b6a-846d-4a4e-a590-9658be0622e6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:51.995675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a57b6a-846d-4a4e-a590-9658be0622e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:52.021974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c3acf54deb947988ed5109422c9ce2f', 'request_id': 'req-76c33440-9872-4bb9-b1d7-fd610c8a1082'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:52.028191 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-74a57b6a-846d-4a4e-a590-9658be0622e6 None None] MFA Rules not processed for user `2149f4b86e064e2e98575720ae43e109`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:52.029474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-74a57b6a-846d-4a4e-a590-9658be0622e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:52.036730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] The token cache is being invalidate because project 8c3acf54deb947988ed5109422c9ce2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:52.037030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:52.037864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-76c33440-9872-4bb9-b1d7-fd610c8a1082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:52.055500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 908/1819] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:51 2022] DELETE /identity/v3/projects/8c3acf54deb947988ed5109422c9ce2f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:52.058757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e441a28-3ead-4551-a818-34e257afac16 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:52.059324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e441a28-3ead-4551-a818-34e257afac16 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:52.059783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e441a28-3ead-4551-a818-34e257afac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:52.117527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-5e441a28-3ead-4551-a818-34e257afac16 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:52.126715 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5e441a28-3ead-4551-a818-34e257afac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:52.186052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 912/1820] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:52.198679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a57b6a-846d-4a4e-a590-9658be0622e6 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:52.200653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-74a57b6a-846d-4a4e-a590-9658be0622e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:52.233189 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 909/1821] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:52.240525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf66109-357c-4a11-b760-200abf40e45b None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:52.240818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf66109-357c-4a11-b760-200abf40e45b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:52.241100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf66109-357c-4a11-b760-200abf40e45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:52.243405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:52.266450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-dbf66109-357c-4a11-b760-200abf40e45b None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:52.266450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-dbf66109-357c-4a11-b760-200abf40e45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:52.274904 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 910/1822] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:52.285764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a57b6a-846d-4a4e-a590-9658be0622e6 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:52.287657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac02b6a-c90e-4d24-881e-96bd9d9f55d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:52.288296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac02b6a-c90e-4d24-881e-96bd9d9f55d0 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:52.288606 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac02b6a-c90e-4d24-881e-96bd9d9f55d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:52.288903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac02b6a-c90e-4d24-881e-96bd9d9f55d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:52.291358 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf66109-357c-4a11-b760-200abf40e45b None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:52.298916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac02b6a-c90e-4d24-881e-96bd9d9f55d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:52.300352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac02b6a-c90e-4d24-881e-96bd9d9f55d0 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:52.300768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf66109-357c-4a11-b760-200abf40e45b None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:52.305673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd3cb6-81c9-4f31-b215-75532c028d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:52.306328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd3cb6-81c9-4f31-b215-75532c028d09 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:52.306633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd3cb6-81c9-4f31-b215-75532c028d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:52.307722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd3cb6-81c9-4f31-b215-75532c028d09 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:52.308642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd3cb6-81c9-4f31-b215-75532c028d09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:52.308953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd3cb6-81c9-4f31-b215-75532c028d09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:52.310652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd3cb6-81c9-4f31-b215-75532c028d09 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:52.315914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 913/1823] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:52.324063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 911/1824] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:52.333232 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac02b6a-c90e-4d24-881e-96bd9d9f55d0 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:52.354712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac02b6a-c90e-4d24-881e-96bd9d9f55d0 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:52.360072 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6545b35-827b-4ee1-80f1-533e3167e7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:52.360932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6545b35-827b-4ee1-80f1-533e3167e7fd admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:52.374256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6545b35-827b-4ee1-80f1-533e3167e7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:52.374890 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6545b35-827b-4ee1-80f1-533e3167e7fd admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:52.375977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6545b35-827b-4ee1-80f1-533e3167e7fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:52.387590 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6545b35-827b-4ee1-80f1-533e3167e7fd admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:52.522169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 914/1825] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:52 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:52.522269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd3cb6-81c9-4f31-b215-75532c028d09 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:52.526412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd3cb6-81c9-4f31-b215-75532c028d09 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:52.627189 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc1d66-f90f-4790-be81-b4cd36d0c3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:52.627845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc1d66-f90f-4790-be81-b4cd36d0c3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:52.628117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc1d66-f90f-4790-be81-b4cd36d0c3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:52.630267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc1d66-f90f-4790-be81-b4cd36d0c3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:52.633970 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc1d66-f90f-4790-be81-b4cd36d0c3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:52.636518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc1d66-f90f-4790-be81-b4cd36d0c3a8 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:52.673793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:52.766983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 912/1826] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:52 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:52.771181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6545b35-827b-4ee1-80f1-533e3167e7fd admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:52.783253 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6545b35-827b-4ee1-80f1-533e3167e7fd admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:52.788670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c29de-5f2d-4e9c-9a6c-89688363081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:52.789366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c29de-5f2d-4e9c-9a6c-89688363081d admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:52.789630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c29de-5f2d-4e9c-9a6c-89688363081d admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:52.789903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c29de-5f2d-4e9c-9a6c-89688363081d admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:52.790675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c29de-5f2d-4e9c-9a6c-89688363081d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:52.793132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c29de-5f2d-4e9c-9a6c-89688363081d admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:52.803362 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 915/1827] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:52.811556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc1d66-f90f-4790-be81-b4cd36d0c3a8 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:52.824178 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc1d66-f90f-4790-be81-b4cd36d0c3a8 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:52.828885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709200b-57ba-4e60-8a77-9f6c501ddac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:52.829513 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709200b-57ba-4e60-8a77-9f6c501ddac2 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:52.829796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709200b-57ba-4e60-8a77-9f6c501ddac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:52.830103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709200b-57ba-4e60-8a77-9f6c501ddac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:52.830803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709200b-57ba-4e60-8a77-9f6c501ddac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:52.832431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709200b-57ba-4e60-8a77-9f6c501ddac2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:52.857984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 913/1828] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:52.861895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c29de-5f2d-4e9c-9a6c-89688363081d admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:52.873165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c29de-5f2d-4e9c-9a6c-89688363081d admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:52.877597 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63fc37-b9ed-4714-8f34-26e37220dd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:52.878289 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63fc37-b9ed-4714-8f34-26e37220dd41 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:52.878572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63fc37-b9ed-4714-8f34-26e37220dd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:52.878846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63fc37-b9ed-4714-8f34-26e37220dd41 admin admin] PATH_INFO: `/v3/projects/33f2f4670ed94c098ecbe33962cb56cb/users/a918b32ad6a3460691aeee2d47fb9117/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:52.893299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63fc37-b9ed-4714-8f34-26e37220dd41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33f2f4670ed94c098ecbe33962cb56cb, user_id=a918b32ad6a3460691aeee2d47fb9117, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:52.896054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63fc37-b9ed-4714-8f34-26e37220dd41 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:52.968865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 916/1829] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:52 2022] PUT /identity/v3/projects/33f2f4670ed94c098ecbe33962cb56cb/users/a918b32ad6a3460691aeee2d47fb9117/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:52.980648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4becb6-89a5-4b19-b773-4318a226a8fe None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:52.981006 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4becb6-89a5-4b19-b773-4318a226a8fe None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:52.981357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4becb6-89a5-4b19-b773-4318a226a8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:53.012608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-9e4becb6-89a5-4b19-b773-4318a226a8fe None None] MFA Rules not processed for user `a918b32ad6a3460691aeee2d47fb9117`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:53.017122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9e4becb6-89a5-4b19-b773-4318a226a8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:53.093385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 914/1830] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:53.096180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63fc37-b9ed-4714-8f34-26e37220dd41 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:53.188231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63fc37-b9ed-4714-8f34-26e37220dd41 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:53.193063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9387711-a595-450e-a97a-d95b386918d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:53.193682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9387711-a595-450e-a97a-d95b386918d2 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:53.193961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9387711-a595-450e-a97a-d95b386918d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:53.194284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9387711-a595-450e-a97a-d95b386918d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:53.194918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9387711-a595-450e-a97a-d95b386918d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:53.196982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9387711-a595-450e-a97a-d95b386918d2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:53.342380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 917/1831] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:53 2022] POST /identity/v3/projects => generated 387 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:53.356607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4becb6-89a5-4b19-b773-4318a226a8fe None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:53.371510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4becb6-89a5-4b19-b773-4318a226a8fe None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:53.429997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46ff24-16ac-49cd-90ef-33d0c9881a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:53.430644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46ff24-16ac-49cd-90ef-33d0c9881a3c admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:53.430926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46ff24-16ac-49cd-90ef-33d0c9881a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:53.431233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46ff24-16ac-49cd-90ef-33d0c9881a3c admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:53.432517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46ff24-16ac-49cd-90ef-33d0c9881a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:53.434460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46ff24-16ac-49cd-90ef-33d0c9881a3c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:53.548654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 915/1832] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:53 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:53.552565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9387711-a595-450e-a97a-d95b386918d2 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:53.563474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9387711-a595-450e-a97a-d95b386918d2 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:53.568125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0e824-aa08-45f7-a73c-b5fbd6a5f0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:53.568776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0e824-aa08-45f7-a73c-b5fbd6a5f0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:53.569058 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0e824-aa08-45f7-a73c-b5fbd6a5f0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:53.569334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0e824-aa08-45f7-a73c-b5fbd6a5f0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:53.570110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0e824-aa08-45f7-a73c-b5fbd6a5f0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:53.571734 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0e824-aa08-45f7-a73c-b5fbd6a5f0d7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:53.584231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 918/1833] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:53.590650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0e824-aa08-45f7-a73c-b5fbd6a5f0d7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:53.590798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46ff24-16ac-49cd-90ef-33d0c9881a3c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:53.603681 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0e824-aa08-45f7-a73c-b5fbd6a5f0d7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:53.609330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e108a7-720b-4dfc-80aa-815360c01c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:53.610076 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e108a7-720b-4dfc-80aa-815360c01c8a admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:53.611039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e108a7-720b-4dfc-80aa-815360c01c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:53.611468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e108a7-720b-4dfc-80aa-815360c01c8a admin admin] PATH_INFO: `/v3/projects/55c1502f3d03414e8a2c46ef88bed770/users/70859c3f905341d9af1f54b886467af1/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:53.629342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e108a7-720b-4dfc-80aa-815360c01c8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c1502f3d03414e8a2c46ef88bed770, user_id=70859c3f905341d9af1f54b886467af1, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:53.632329 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e108a7-720b-4dfc-80aa-815360c01c8a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:53.665519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46ff24-16ac-49cd-90ef-33d0c9881a3c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:53.670375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5299a8-0554-45d0-a873-f327eb6e78ac tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2149f4b86e064e2e98575720ae43e109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12d99dc2e48e41179cbe98a2e3c69d2d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:53.670990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5299a8-0554-45d0-a873-f327eb6e78ac tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:53.671326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5299a8-0554-45d0-a873-f327eb6e78ac tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:53.671594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5299a8-0554-45d0-a873-f327eb6e78ac tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:53.672289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5299a8-0554-45d0-a873-f327eb6e78ac tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:53.672598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5299a8-0554-45d0-a873-f327eb6e78ac tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:53.674216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5299a8-0554-45d0-a873-f327eb6e78ac tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:53.687207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 916/1834] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:53.691347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5299a8-0554-45d0-a873-f327eb6e78ac tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:53.701687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5299a8-0554-45d0-a873-f327eb6e78ac tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:53.706588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee143854-7f6e-4a9d-9bfc-ef0a02fded63 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2149f4b86e064e2e98575720ae43e109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12d99dc2e48e41179cbe98a2e3c69d2d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:53.707195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee143854-7f6e-4a9d-9bfc-ef0a02fded63 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:53.707478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee143854-7f6e-4a9d-9bfc-ef0a02fded63 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:53.707757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee143854-7f6e-4a9d-9bfc-ef0a02fded63 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:53.708422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee143854-7f6e-4a9d-9bfc-ef0a02fded63 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:53.710335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee143854-7f6e-4a9d-9bfc-ef0a02fded63 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:53.723000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 919/1835] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:53 2022] PUT /identity/v3/projects/55c1502f3d03414e8a2c46ef88bed770/users/70859c3f905341d9af1f54b886467af1/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:53.725839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e108a7-720b-4dfc-80aa-815360c01c8a admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:53.789417 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e108a7-720b-4dfc-80aa-815360c01c8a admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:53.794918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bb465-3a4d-454b-b1f5-5644eaa57f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:53.795550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bb465-3a4d-454b-b1f5-5644eaa57f1f admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:53.795836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bb465-3a4d-454b-b1f5-5644eaa57f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:53.796124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bb465-3a4d-454b-b1f5-5644eaa57f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:53.796845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080bb465-3a4d-454b-b1f5-5644eaa57f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:53.798558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080bb465-3a4d-454b-b1f5-5644eaa57f1f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:53.801228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 917/1836] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:53 2022] POST /identity/v3/projects => generated 353 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:53.804916 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee143854-7f6e-4a9d-9bfc-ef0a02fded63 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:53.815632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 920/1837] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:53.819494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bb465-3a4d-454b-b1f5-5644eaa57f1f admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:53.829786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bb465-3a4d-454b-b1f5-5644eaa57f1f admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:53.887195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee143854-7f6e-4a9d-9bfc-ef0a02fded63 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:53.891889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e652d3-e85b-499a-92a8-033fee51bb12 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2149f4b86e064e2e98575720ae43e109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12d99dc2e48e41179cbe98a2e3c69d2d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:53.892504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e652d3-e85b-499a-92a8-033fee51bb12 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:53.892841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e652d3-e85b-499a-92a8-033fee51bb12 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:53.893068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e652d3-e85b-499a-92a8-033fee51bb12 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:53.893767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e652d3-e85b-499a-92a8-033fee51bb12 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:53.894129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e652d3-e85b-499a-92a8-033fee51bb12 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:53.894245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949236ee-b164-4714-b209-aaa96eaf5931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:53.894892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949236ee-b164-4714-b209-aaa96eaf5931 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:53.895178 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949236ee-b164-4714-b209-aaa96eaf5931 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:53.895455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949236ee-b164-4714-b209-aaa96eaf5931 admin admin] PATH_INFO: `/v3/projects/55c1502f3d03414e8a2c46ef88bed770/users/70859c3f905341d9af1f54b886467af1/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:53.895748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e652d3-e85b-499a-92a8-033fee51bb12 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:53.898867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949236ee-b164-4714-b209-aaa96eaf5931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c1502f3d03414e8a2c46ef88bed770, user_id=70859c3f905341d9af1f54b886467af1, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:53.902516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949236ee-b164-4714-b209-aaa96eaf5931 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:53.908460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 918/1838] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:40:53.916068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 921/1839] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:53 2022] PUT /identity/v3/projects/55c1502f3d03414e8a2c46ef88bed770/users/70859c3f905341d9af1f54b886467af1/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:53.927106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a86d7-228a-4d14-ae75-4bdabdc08d57 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:53.927404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a86d7-228a-4d14-ae75-4bdabdc08d57 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:53.927684 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a86d7-228a-4d14-ae75-4bdabdc08d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:53.954495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949236ee-b164-4714-b209-aaa96eaf5931 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:53.962254 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-293a86d7-228a-4d14-ae75-4bdabdc08d57 None None] MFA Rules not processed for user `70859c3f905341d9af1f54b886467af1`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:53.963571 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-293a86d7-228a-4d14-ae75-4bdabdc08d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:53.970993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:54.035008 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:54.040480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949236ee-b164-4714-b209-aaa96eaf5931 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:54.046451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2149f4b86e064e2e98575720ae43e109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12d99dc2e48e41179cbe98a2e3c69d2d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:54.047197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:54.047728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:54.048603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] PATH_INFO: `/v3/projects/1ebbec259afb422ba70d0d8f1e15869e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:54.049864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1ebbec259afb422ba70d0d8f1e15869e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:54.059968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:54.063909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 919/1840] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:54.079908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a86d7-228a-4d14-ae75-4bdabdc08d57 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:54.148640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a86d7-228a-4d14-ae75-4bdabdc08d57 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:54.152931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965888f-ce81-40d6-b9a8-a979da3c7cc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:54.153562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965888f-ce81-40d6-b9a8-a979da3c7cc7 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:54.153846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965888f-ce81-40d6-b9a8-a979da3c7cc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:54.154185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965888f-ce81-40d6-b9a8-a979da3c7cc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:54.165354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965888f-ce81-40d6-b9a8-a979da3c7cc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:54.167159 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965888f-ce81-40d6-b9a8-a979da3c7cc7 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:54.171537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ebbec259afb422ba70d0d8f1e15869e', 'request_id': 'req-efc8d96a-312e-474b-9e45-5fd72d686643'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:54.177415 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 920/1841] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:40:54.222737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965888f-ce81-40d6-b9a8-a979da3c7cc7 service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:54.241283 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ebbec259afb422ba70d0d8f1e15869e', 'request_id': 'req-efc8d96a-312e-474b-9e45-5fd72d686643'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:54.248666 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ebbec259afb422ba70d0d8f1e15869e', 'request_id': 'req-efc8d96a-312e-474b-9e45-5fd72d686643'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:54.261495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] The token cache is being invalidate because project 1ebbec259afb422ba70d0d8f1e15869e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:54.261793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:54.262532 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-efc8d96a-312e-474b-9e45-5fd72d686643 tempest-VolumeQuotaClassesTest-282696191 tempest-VolumeQuotaClassesTest-282696191-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:54.264365 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 922/1842] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:53 2022] DELETE /identity/v3/projects/1ebbec259afb422ba70d0d8f1e15869e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:54.304645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965888f-ce81-40d6-b9a8-a979da3c7cc7 service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:54.393794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:54.394543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:54.394827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:54.395102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] PATH_INFO: `/v3/users/a918b32ad6a3460691aeee2d47fb9117` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:54.396324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a918b32ad6a3460691aeee2d47fb9117)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:54.399301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:54.694448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a918b32ad6a3460691aeee2d47fb9117'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:54.743932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a918b32ad6a3460691aeee2d47fb9117'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:54.745542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a918b32ad6a3460691aeee2d47fb9117'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:54.758703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a918b32ad6a3460691aeee2d47fb9117'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:54.766638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a918b32ad6a3460691aeee2d47fb9117'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:54.782319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 921/1843] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:54 2022] DELETE /identity/v3/users/a918b32ad6a3460691aeee2d47fb9117 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:54.786942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0da09-d4b8-48ec-99d3-1a7f435576d4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:54.787230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0da09-d4b8-48ec-99d3-1a7f435576d4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:54.787535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0da09-d4b8-48ec-99d3-1a7f435576d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:54.807803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-eec0da09-d4b8-48ec-99d3-1a7f435576d4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:54.809275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-eec0da09-d4b8-48ec-99d3-1a7f435576d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:54.880185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 923/1844] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:54.896143 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:54.897599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:54.973559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dabb8-f955-45e2-84eb-3d488b71d3e8 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:54.977924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8acdbf-b2e7-4285-b9bd-bd01aaea2087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:54.979290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8acdbf-b2e7-4285-b9bd-bd01aaea2087 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:54.979570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8acdbf-b2e7-4285-b9bd-bd01aaea2087 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:54.980577 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8acdbf-b2e7-4285-b9bd-bd01aaea2087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:54.990260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8acdbf-b2e7-4285-b9bd-bd01aaea2087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:54.998921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8acdbf-b2e7-4285-b9bd-bd01aaea2087 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:55.028377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:55.032474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 922/1845] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:55.033955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0da09-d4b8-48ec-99d3-1a7f435576d4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:55.035395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-eec0da09-d4b8-48ec-99d3-1a7f435576d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:55.049179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0da09-d4b8-48ec-99d3-1a7f435576d4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:55.053922 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:55.054614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:55.054896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:55.055168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] PATH_INFO: `/v3/users/c351fe4452ee4561a6c087f40b8b2366` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:55.056355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c351fe4452ee4561a6c087f40b8b2366)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:55.058533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:55.298512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c351fe4452ee4561a6c087f40b8b2366'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:55.404465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c351fe4452ee4561a6c087f40b8b2366'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:55.418641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c351fe4452ee4561a6c087f40b8b2366'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:55.419356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c351fe4452ee4561a6c087f40b8b2366'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:55.427145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c351fe4452ee4561a6c087f40b8b2366'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:55.437369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 924/1846] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:55 2022] DELETE /identity/v3/users/c351fe4452ee4561a6c087f40b8b2366 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:55.439051 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a858-184c-4f4b-a4d3-b19c1a8b8562 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:55.439554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a858-184c-4f4b-a4d3-b19c1a8b8562 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:55.439757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a858-184c-4f4b-a4d3-b19c1a8b8562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:55.460079 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-6775a858-184c-4f4b-a4d3-b19c1a8b8562 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:55.461385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6775a858-184c-4f4b-a4d3-b19c1a8b8562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:55.530874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 923/1847] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:55.539817 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:55.542223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:55.597550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775a858-184c-4f4b-a4d3-b19c1a8b8562 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:55.599011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6775a858-184c-4f4b-a4d3-b19c1a8b8562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:55.610908 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45ace6-d271-452a-b988-528b1ef31958 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:55.618028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775a858-184c-4f4b-a4d3-b19c1a8b8562 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:55.624169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030da5ac-510c-474e-8cf6-36faadcd57eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:55.624977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030da5ac-510c-474e-8cf6-36faadcd57eb service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:55.625366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030da5ac-510c-474e-8cf6-36faadcd57eb service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:55.625742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030da5ac-510c-474e-8cf6-36faadcd57eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:55.626613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:55.627840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:55.628362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:55.628726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] PATH_INFO: `/v3/projects/33f2f4670ed94c098ecbe33962cb56cb` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:55.630384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=33f2f4670ed94c098ecbe33962cb56cb)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:55.632339 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:55.641331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030da5ac-510c-474e-8cf6-36faadcd57eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:55.642813 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030da5ac-510c-474e-8cf6-36faadcd57eb service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:55.655390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 925/1848] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:40:55.659495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:55.822449 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33f2f4670ed94c098ecbe33962cb56cb', 'request_id': 'req-967b76ca-15b6-451c-947c-13cd57521bfa'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:55.840066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33f2f4670ed94c098ecbe33962cb56cb', 'request_id': 'req-967b76ca-15b6-451c-947c-13cd57521bfa'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:55.908985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33f2f4670ed94c098ecbe33962cb56cb', 'request_id': 'req-967b76ca-15b6-451c-947c-13cd57521bfa'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:55.922741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] The token cache is being invalidate because project 33f2f4670ed94c098ecbe33962cb56cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:55.923394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:55.924353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:55.926465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 924/1849] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:55 2022] DELETE /identity/v3/projects/33f2f4670ed94c098ecbe33962cb56cb => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:55.930336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030da5ac-510c-474e-8cf6-36faadcd57eb service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:55.940801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030da5ac-510c-474e-8cf6-36faadcd57eb service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:56.004640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:56.005284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:56.005554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:56.005832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] PATH_INFO: `/v3/users/70859c3f905341d9af1f54b886467af1` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:56.007173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=70859c3f905341d9af1f54b886467af1)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:56.009292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:56.132482 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:56.157681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:56.159171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:56.172851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b76ca-15b6-451c-947c-13cd57521bfa admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:56.177713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:56.178649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:56.179120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:56.179618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] PATH_INFO: `/v3/projects/c45276ef762c4b77bc7953a6bd74d983` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:56.181025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c45276ef762c4b77bc7953a6bd74d983)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:56.183136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:56.252343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70859c3f905341d9af1f54b886467af1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.287352 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c45276ef762c4b77bc7953a6bd74d983', 'request_id': 'req-efc6d922-9925-4aeb-859d-418a47631ae7'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.295785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c45276ef762c4b77bc7953a6bd74d983', 'request_id': 'req-efc6d922-9925-4aeb-859d-418a47631ae7'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.347383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70859c3f905341d9af1f54b886467af1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.360953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70859c3f905341d9af1f54b886467af1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.361938 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70859c3f905341d9af1f54b886467af1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.380364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70859c3f905341d9af1f54b886467af1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.387858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 926/1850] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:55 2022] DELETE /identity/v3/users/70859c3f905341d9af1f54b886467af1 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:56.433434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c45276ef762c4b77bc7953a6bd74d983', 'request_id': 'req-efc6d922-9925-4aeb-859d-418a47631ae7'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.446613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] The token cache is being invalidate because project c45276ef762c4b77bc7953a6bd74d983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:56.448525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.449248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.450997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 925/1851] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:56 2022] DELETE /identity/v3/projects/c45276ef762c4b77bc7953a6bd74d983 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:40:56.455116 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:56.456993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:56.529772 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ed328-b880-4c05-8e6d-4a29169762bd admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:56.535055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200da495-dafa-459b-9b5e-2a8b6d837e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:56.535859 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200da495-dafa-459b-9b5e-2a8b6d837e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:56.536263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200da495-dafa-459b-9b5e-2a8b6d837e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:56.536564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200da495-dafa-459b-9b5e-2a8b6d837e09 admin admin] PATH_INFO: `/v3/users/2149f4b86e064e2e98575720ae43e109` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:56.537999 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200da495-dafa-459b-9b5e-2a8b6d837e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2149f4b86e064e2e98575720ae43e109)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:56.539931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200da495-dafa-459b-9b5e-2a8b6d837e09 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:56.727105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:56.738387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-200da495-dafa-459b-9b5e-2a8b6d837e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2149f4b86e064e2e98575720ae43e109'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.824925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-200da495-dafa-459b-9b5e-2a8b6d837e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2149f4b86e064e2e98575720ae43e109'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.840037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-200da495-dafa-459b-9b5e-2a8b6d837e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2149f4b86e064e2e98575720ae43e109'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.840837 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-200da495-dafa-459b-9b5e-2a8b6d837e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2149f4b86e064e2e98575720ae43e109'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.848334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-200da495-dafa-459b-9b5e-2a8b6d837e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2149f4b86e064e2e98575720ae43e109'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:56.855268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 927/1852] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:40:56 2022] DELETE /identity/v3/users/2149f4b86e064e2e98575720ae43e109 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:56.927590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:56.929269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:57.058216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6d922-9925-4aeb-859d-418a47631ae7 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:57.138657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:57.145038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:57.145802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:57.146185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:57.146530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] PATH_INFO: `/v3/projects/55c1502f3d03414e8a2c46ef88bed770` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:57.147846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c1502f3d03414e8a2c46ef88bed770)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:57.150769 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:57.281522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c1502f3d03414e8a2c46ef88bed770', 'request_id': 'req-59c32720-c03a-4067-8033-4ee522b04c77'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:57.287988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c1502f3d03414e8a2c46ef88bed770', 'request_id': 'req-59c32720-c03a-4067-8033-4ee522b04c77'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:57.372032 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c1502f3d03414e8a2c46ef88bed770', 'request_id': 'req-59c32720-c03a-4067-8033-4ee522b04c77'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:57.389395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] The token cache is being invalidate because project 55c1502f3d03414e8a2c46ef88bed770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:57.391883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:57.393189 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:57.399446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 926/1853] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:56 2022] DELETE /identity/v3/projects/55c1502f3d03414e8a2c46ef88bed770 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:57.406117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab7c31-d19d-4f01-a86c-c00f81273d97 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:57.407098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab7c31-d19d-4f01-a86c-c00f81273d97 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:57.407616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab7c31-d19d-4f01-a86c-c00f81273d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:57.456047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-25ab7c31-d19d-4f01-a86c-c00f81273d97 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:57.457424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-25ab7c31-d19d-4f01-a86c-c00f81273d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:57.530549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:57.532312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:57.571881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 928/1854] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:57.580103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900729f-0733-48b2-9fa8-7109ca287d09 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:57.580400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900729f-0733-48b2-9fa8-7109ca287d09 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:57.580671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900729f-0733-48b2-9fa8-7109ca287d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:57.580925 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:57.621430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1900729f-0733-48b2-9fa8-7109ca287d09 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:57.623522 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1900729f-0733-48b2-9fa8-7109ca287d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:57.646068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 929/1855] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:57.650798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:57.651396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900729f-0733-48b2-9fa8-7109ca287d09 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:57.662233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c32720-c03a-4067-8033-4ee522b04c77 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:57.664496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900729f-0733-48b2-9fa8-7109ca287d09 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:57.668462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:57.669522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:57.670296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:57.670722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] PATH_INFO: `/v3/projects/12d99dc2e48e41179cbe98a2e3c69d2d` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:57.670832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bfe1d-0d4a-43d0-a9d8-467526c8a823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:57.671428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bfe1d-0d4a-43d0-a9d8-467526c8a823 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:57.671708 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bfe1d-0d4a-43d0-a9d8-467526c8a823 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:57.671981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bfe1d-0d4a-43d0-a9d8-467526c8a823 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:57.672793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12d99dc2e48e41179cbe98a2e3c69d2d)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:57.673028 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bfe1d-0d4a-43d0-a9d8-467526c8a823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:57.673471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bfe1d-0d4a-43d0-a9d8-467526c8a823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:57.675457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:57.675750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bfe1d-0d4a-43d0-a9d8-467526c8a823 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:57.691710 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 930/1856] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:40:57.701743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bfe1d-0d4a-43d0-a9d8-467526c8a823 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:57.719656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bfe1d-0d4a-43d0-a9d8-467526c8a823 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:57.725022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d1a2b-1675-4f4e-b160-ed20466acaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:57.725696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d1a2b-1675-4f4e-b160-ed20466acaed admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:57.726079 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d1a2b-1675-4f4e-b160-ed20466acaed admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:57.726421 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d1a2b-1675-4f4e-b160-ed20466acaed admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:57.727161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d1a2b-1675-4f4e-b160-ed20466acaed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:57.729522 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d1a2b-1675-4f4e-b160-ed20466acaed admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:57.852347 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 931/1857] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:57 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:57.856474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d1a2b-1675-4f4e-b160-ed20466acaed admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:57.867291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12d99dc2e48e41179cbe98a2e3c69d2d', 'request_id': 'req-897fd105-1abf-4b3d-8283-a1e9385f0816'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:57.880170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12d99dc2e48e41179cbe98a2e3c69d2d', 'request_id': 'req-897fd105-1abf-4b3d-8283-a1e9385f0816'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:57.949486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12d99dc2e48e41179cbe98a2e3c69d2d', 'request_id': 'req-897fd105-1abf-4b3d-8283-a1e9385f0816'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:57.955845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d1a2b-1675-4f4e-b160-ed20466acaed admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:57.962505 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cb7b3-5c30-4e1d-82df-50949d9d1f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:57.964453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cb7b3-5c30-4e1d-82df-50949d9d1f14 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:57.965069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cb7b3-5c30-4e1d-82df-50949d9d1f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:57.965664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cb7b3-5c30-4e1d-82df-50949d9d1f14 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:57.966759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cb7b3-5c30-4e1d-82df-50949d9d1f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:57.968051 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] The token cache is being invalidate because project 12d99dc2e48e41179cbe98a2e3c69d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:40:57.968316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:57.968959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cb7b3-5c30-4e1d-82df-50949d9d1f14 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:57.974629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-897fd105-1abf-4b3d-8283-a1e9385f0816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:40:57.990085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 927/1858] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:40:57 2022] DELETE /identity/v3/projects/12d99dc2e48e41179cbe98a2e3c69d2d => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:57.991572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d998b2c-cad8-447d-a6bc-ab0fcc45aa03 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:57.992170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d998b2c-cad8-447d-a6bc-ab0fcc45aa03 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:57.992462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d998b2c-cad8-447d-a6bc-ab0fcc45aa03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.015239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-8d998b2c-cad8-447d-a6bc-ab0fcc45aa03 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:58.016503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8d998b2c-cad8-447d-a6bc-ab0fcc45aa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:58.025090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 928/1859] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:40:58.033374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22653db-f884-40c1-8add-ae3cb0c5c115 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.033667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22653db-f884-40c1-8add-ae3cb0c5c115 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.033951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22653db-f884-40c1-8add-ae3cb0c5c115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.059216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-b22653db-f884-40c1-8add-ae3cb0c5c115 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:58.060483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b22653db-f884-40c1-8add-ae3cb0c5c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:58.074939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 932/1860] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:57 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:58.085364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cb7b3-5c30-4e1d-82df-50949d9d1f14 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:58.087607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-350cb7b3-5c30-4e1d-82df-50949d9d1f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:58.091531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 929/1861] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:58.097327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22653db-f884-40c1-8add-ae3cb0c5c115 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:58.100953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cb7b3-5c30-4e1d-82df-50949d9d1f14 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:58.107191 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a913884-0379-4eda-902a-93286ab0a2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:58.107814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a913884-0379-4eda-902a-93286ab0a2be admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.108092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a913884-0379-4eda-902a-93286ab0a2be admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.109130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a913884-0379-4eda-902a-93286ab0a2be admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.109130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a913884-0379-4eda-902a-93286ab0a2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:58.110657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a913884-0379-4eda-902a-93286ab0a2be admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:58.111770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22653db-f884-40c1-8add-ae3cb0c5c115 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:58.120517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e258-5322-46aa-a8cf-31b4e865303a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:58.121149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e258-5322-46aa-a8cf-31b4e865303a admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.121438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e258-5322-46aa-a8cf-31b4e865303a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.121708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e258-5322-46aa-a8cf-31b4e865303a admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.122468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235e258-5322-46aa-a8cf-31b4e865303a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:40:58.122777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235e258-5322-46aa-a8cf-31b4e865303a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:58.126229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 933/1862] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:58.126374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235e258-5322-46aa-a8cf-31b4e865303a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:58.128952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a913884-0379-4eda-902a-93286ab0a2be admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:58.140397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a913884-0379-4eda-902a-93286ab0a2be admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:58.145368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16afaa-a429-4cbe-a9a8-4fb2d9365787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:58.151308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 930/1863] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:40:58.152528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16afaa-a429-4cbe-a9a8-4fb2d9365787 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.153482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16afaa-a429-4cbe-a9a8-4fb2d9365787 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.153876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16afaa-a429-4cbe-a9a8-4fb2d9365787 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.155274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16afaa-a429-4cbe-a9a8-4fb2d9365787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:58.158682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e258-5322-46aa-a8cf-31b4e865303a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:58.160665 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16afaa-a429-4cbe-a9a8-4fb2d9365787 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:58.186494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e258-5322-46aa-a8cf-31b4e865303a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:58.187336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 934/1864] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:58.194986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e383b-8148-4270-bf0a-a09a22a9032f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:58.195313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16afaa-a429-4cbe-a9a8-4fb2d9365787 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:58.195625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e383b-8148-4270-bf0a-a09a22a9032f admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.195909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e383b-8148-4270-bf0a-a09a22a9032f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.198877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e383b-8148-4270-bf0a-a09a22a9032f admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.198877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e383b-8148-4270-bf0a-a09a22a9032f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:58.208077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16afaa-a429-4cbe-a9a8-4fb2d9365787 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:58.208952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e383b-8148-4270-bf0a-a09a22a9032f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:58.214528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b86e6-64a9-4976-a7c5-cbe3bfd83af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:58.215303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b86e6-64a9-4976-a7c5-cbe3bfd83af7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.215664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b86e6-64a9-4976-a7c5-cbe3bfd83af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.215996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b86e6-64a9-4976-a7c5-cbe3bfd83af7 admin admin] PATH_INFO: `/v3/projects/b7f2a60a8bd04ec092e5794dbbf48b84/users/851ecae88d3c440db2fc68ab16a700ee/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.242246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b86e6-64a9-4976-a7c5-cbe3bfd83af7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f2a60a8bd04ec092e5794dbbf48b84, user_id=851ecae88d3c440db2fc68ab16a700ee, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:58.247444 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b86e6-64a9-4976-a7c5-cbe3bfd83af7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:58.385833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 935/1865] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:58 2022] PUT /identity/v3/projects/b7f2a60a8bd04ec092e5794dbbf48b84/users/851ecae88d3c440db2fc68ab16a700ee/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:58.400160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3254c4d-0ae3-45f2-aa71-730602bdf55d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.401031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3254c4d-0ae3-45f2-aa71-730602bdf55d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.401315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3254c4d-0ae3-45f2-aa71-730602bdf55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.417135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 931/1866] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:58 2022] POST /identity/v3/projects => generated 365 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:58.422363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e383b-8148-4270-bf0a-a09a22a9032f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:58.440943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c3254c4d-0ae3-45f2-aa71-730602bdf55d None None] MFA Rules not processed for user `851ecae88d3c440db2fc68ab16a700ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:58.442474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c3254c4d-0ae3-45f2-aa71-730602bdf55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:58.537335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e383b-8148-4270-bf0a-a09a22a9032f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:58.544297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:58.547690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44835840-2a8b-40d0-a226-e37d55ca9767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:58.548498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44835840-2a8b-40d0-a226-e37d55ca9767 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.548856 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44835840-2a8b-40d0-a226-e37d55ca9767 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.549141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44835840-2a8b-40d0-a226-e37d55ca9767 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.549824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44835840-2a8b-40d0-a226-e37d55ca9767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:58.552551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44835840-2a8b-40d0-a226-e37d55ca9767 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:58.573335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 936/1867] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:58.639824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3254c4d-0ae3-45f2-aa71-730602bdf55d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:58.641517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c3254c4d-0ae3-45f2-aa71-730602bdf55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:58.649086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 932/1868] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:58 2022] POST /identity/v3/users => generated 283 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:40:58.655150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44835840-2a8b-40d0-a226-e37d55ca9767 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:58.672637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44835840-2a8b-40d0-a226-e37d55ca9767 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:58.681390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e8c38-693b-4d88-90b4-d83f3c24531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:58.682275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e8c38-693b-4d88-90b4-d83f3c24531f admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.682546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e8c38-693b-4d88-90b4-d83f3c24531f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.682807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e8c38-693b-4d88-90b4-d83f3c24531f admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.684129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e8c38-693b-4d88-90b4-d83f3c24531f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:58.685139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e8c38-693b-4d88-90b4-d83f3c24531f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:58.710213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 933/1869] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:58.715585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e8c38-693b-4d88-90b4-d83f3c24531f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:58.733909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e8c38-693b-4d88-90b4-d83f3c24531f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:58.750793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993cf03-8ca3-46b6-b2ac-5b00ad85b8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:58.751025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3254c4d-0ae3-45f2-aa71-730602bdf55d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:58.751725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993cf03-8ca3-46b6-b2ac-5b00ad85b8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.752281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993cf03-8ca3-46b6-b2ac-5b00ad85b8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.752666 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993cf03-8ca3-46b6-b2ac-5b00ad85b8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.753674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993cf03-8ca3-46b6-b2ac-5b00ad85b8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:58.755220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993cf03-8ca3-46b6-b2ac-5b00ad85b8e9 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:58.756376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec95f2-d433-4f96-a3e5-4b5f3c2032ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:58.757373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec95f2-d433-4f96-a3e5-4b5f3c2032ba service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.758076 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec95f2-d433-4f96-a3e5-4b5f3c2032ba service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.758362 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec95f2-d433-4f96-a3e5-4b5f3c2032ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.769311 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec95f2-d433-4f96-a3e5-4b5f3c2032ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:58.771025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec95f2-d433-4f96-a3e5-4b5f3c2032ba service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:58.774824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 934/1870] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:40:58.777574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993cf03-8ca3-46b6-b2ac-5b00ad85b8e9 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:58.786284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 937/1871] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:40:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:40:58.799827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993cf03-8ca3-46b6-b2ac-5b00ad85b8e9 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:58.831086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652773e-cfb1-4ddf-bd2d-9d69121d6d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:58.832022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652773e-cfb1-4ddf-bd2d-9d69121d6d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.832258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652773e-cfb1-4ddf-bd2d-9d69121d6d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.832555 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652773e-cfb1-4ddf-bd2d-9d69121d6d06 admin admin] PATH_INFO: `/v3/projects/b3256a49c3524e4e9916d8170460a341/users/b2282f1ab34a472fa918e0f014dfe46d/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:58.860681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652773e-cfb1-4ddf-bd2d-9d69121d6d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3256a49c3524e4e9916d8170460a341, user_id=b2282f1ab34a472fa918e0f014dfe46d, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:58.864973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652773e-cfb1-4ddf-bd2d-9d69121d6d06 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:58.958448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 935/1872] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:40:58 2022] PUT /identity/v3/projects/b3256a49c3524e4e9916d8170460a341/users/b2282f1ab34a472fa918e0f014dfe46d/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:40:58.960194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6740e-62f7-4269-b90e-8420feed8e56 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:58.960556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6740e-62f7-4269-b90e-8420feed8e56 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:58.960864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6740e-62f7-4269-b90e-8420feed8e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:59.004779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1bd6740e-62f7-4269-b90e-8420feed8e56 None None] MFA Rules not processed for user `b2282f1ab34a472fa918e0f014dfe46d`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:40:59.004779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1bd6740e-62f7-4269-b90e-8420feed8e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:40:59.052484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:40:59.115392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 938/1873] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:40:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:40:59.121034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652773e-cfb1-4ddf-bd2d-9d69121d6d06 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:59.288401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652773e-cfb1-4ddf-bd2d-9d69121d6d06 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:40:59.288401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a0d6a-9668-4439-b01d-73aafea4d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:40:59.288401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a0d6a-9668-4439-b01d-73aafea4d14a admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:40:59.288401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a0d6a-9668-4439-b01d-73aafea4d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:40:59.289803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a0d6a-9668-4439-b01d-73aafea4d14a admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:40:59.289803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a0d6a-9668-4439-b01d-73aafea4d14a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:40:59.289803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a0d6a-9668-4439-b01d-73aafea4d14a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:40:59.922932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 936/1874] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:40:59 2022] POST /identity/v3/projects => generated 365 bytes in 793 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:40:59.923603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6740e-62f7-4269-b90e-8420feed8e56 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:40:59.937607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6740e-62f7-4269-b90e-8420feed8e56 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:00.079009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeca54-c2c2-4733-ab03-c2c96fecc1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:00.079399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeca54-c2c2-4733-ab03-c2c96fecc1ee admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:00.079683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeca54-c2c2-4733-ab03-c2c96fecc1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:00.080378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeca54-c2c2-4733-ab03-c2c96fecc1ee admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:00.080903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caeca54-c2c2-4733-ab03-c2c96fecc1ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:00.083023 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caeca54-c2c2-4733-ab03-c2c96fecc1ee admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:01.162029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 939/1875] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:40:59 2022] POST /identity/v3/users => generated 289 bytes in 1244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:41:01.163424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a0d6a-9668-4439-b01d-73aafea4d14a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:01.179752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a0d6a-9668-4439-b01d-73aafea4d14a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:01.186446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef722f-40a8-4bd8-883e-8cba549f0f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:01.187457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef722f-40a8-4bd8-883e-8cba549f0f2f admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:01.187894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef722f-40a8-4bd8-883e-8cba549f0f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:01.188302 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef722f-40a8-4bd8-883e-8cba549f0f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:01.189198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef722f-40a8-4bd8-883e-8cba549f0f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:01.190989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef722f-40a8-4bd8-883e-8cba549f0f2f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:01.203548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 937/1876] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:01.207672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeca54-c2c2-4733-ab03-c2c96fecc1ee admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:01.223195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeca54-c2c2-4733-ab03-c2c96fecc1ee admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:01.229272 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b78204-5e3a-432b-b36b-140c24359bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:01.230148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b78204-5e3a-432b-b36b-140c24359bee admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:01.230435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b78204-5e3a-432b-b36b-140c24359bee admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:01.230799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b78204-5e3a-432b-b36b-140c24359bee admin admin] PATH_INFO: `/v3/projects/e3e10996d343468093654cf19471181d/users/e77906cdf9dd456eac480c66fc2e267f/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:01.248616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b78204-5e3a-432b-b36b-140c24359bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e10996d343468093654cf19471181d, user_id=e77906cdf9dd456eac480c66fc2e267f, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:01.251389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b78204-5e3a-432b-b36b-140c24359bee admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:01.374133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 940/1877] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:01 2022] PUT /identity/v3/projects/e3e10996d343468093654cf19471181d/users/e77906cdf9dd456eac480c66fc2e267f/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:41:01.381368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef722f-40a8-4bd8-883e-8cba549f0f2f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:01.412062 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef722f-40a8-4bd8-883e-8cba549f0f2f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:01.542836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95ced8-09c6-4533-bd92-197e2cbe19bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:01.544611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95ced8-09c6-4533-bd92-197e2cbe19bd admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:01.545294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95ced8-09c6-4533-bd92-197e2cbe19bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:01.545674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95ced8-09c6-4533-bd92-197e2cbe19bd admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:01.546528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95ced8-09c6-4533-bd92-197e2cbe19bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:01.548459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95ced8-09c6-4533-bd92-197e2cbe19bd admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:01.560222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 938/1878] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:01.564114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b78204-5e3a-432b-b36b-140c24359bee admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:01.580390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b78204-5e3a-432b-b36b-140c24359bee admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:01.591907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ec8a1-f38f-421d-9cd7-6721dd814846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:01.597728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ec8a1-f38f-421d-9cd7-6721dd814846 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:01.597728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ec8a1-f38f-421d-9cd7-6721dd814846 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:01.597728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ec8a1-f38f-421d-9cd7-6721dd814846 admin admin] PATH_INFO: `/v3/projects/e3e10996d343468093654cf19471181d/users/e77906cdf9dd456eac480c66fc2e267f/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:01.605094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ec8a1-f38f-421d-9cd7-6721dd814846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e10996d343468093654cf19471181d, user_id=e77906cdf9dd456eac480c66fc2e267f, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:01.611210 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ec8a1-f38f-421d-9cd7-6721dd814846 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:01.650592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 941/1879] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:01 2022] PUT /identity/v3/projects/e3e10996d343468093654cf19471181d/users/e77906cdf9dd456eac480c66fc2e267f/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:01.663498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee10ae3-9f9d-445f-8fc4-bb7b3e511425 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:01.663787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee10ae3-9f9d-445f-8fc4-bb7b3e511425 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:01.664186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee10ae3-9f9d-445f-8fc4-bb7b3e511425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:01.711836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-fee10ae3-9f9d-445f-8fc4-bb7b3e511425 None None] MFA Rules not processed for user `e77906cdf9dd456eac480c66fc2e267f`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:01.712424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-fee10ae3-9f9d-445f-8fc4-bb7b3e511425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:01.819242 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 939/1880] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:01.827575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ec8a1-f38f-421d-9cd7-6721dd814846 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:01.903935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ec8a1-f38f-421d-9cd7-6721dd814846 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:01.909171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc3914-becc-44fb-80d3-20aad9902114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:01.909171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc3914-becc-44fb-80d3-20aad9902114 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:01.909171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc3914-becc-44fb-80d3-20aad9902114 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:01.909171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc3914-becc-44fb-80d3-20aad9902114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:01.922580 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc3914-becc-44fb-80d3-20aad9902114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:01.924265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc3914-becc-44fb-80d3-20aad9902114 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:01.936873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 942/1881] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:02.170831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee10ae3-9f9d-445f-8fc4-bb7b3e511425 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:02.173559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:02.174534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-fee10ae3-9f9d-445f-8fc4-bb7b3e511425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:02.186838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee10ae3-9f9d-445f-8fc4-bb7b3e511425 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:02.190614 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5771c-2c8b-4e95-af2a-106afc7d0531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:02.191327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5771c-2c8b-4e95-af2a-106afc7d0531 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:02.191678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5771c-2c8b-4e95-af2a-106afc7d0531 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:02.192023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5771c-2c8b-4e95-af2a-106afc7d0531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:02.257309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5771c-2c8b-4e95-af2a-106afc7d0531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:02.258897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5771c-2c8b-4e95-af2a-106afc7d0531 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:02.276611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 940/1882] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:41:03.115458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:03.724310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc3914-becc-44fb-80d3-20aad9902114 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:03.727450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-dacc3914-becc-44fb-80d3-20aad9902114 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:03.799935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc3914-becc-44fb-80d3-20aad9902114 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:03.804653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:03.805233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:03.805517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:03.805792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] PATH_INFO: `/v3/users/8249aca8c5b74f54945f04094d348057` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:03.807685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8249aca8c5b74f54945f04094d348057)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:03.809656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:04.013704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8249aca8c5b74f54945f04094d348057'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:04.074532 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8249aca8c5b74f54945f04094d348057'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:04.087660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8249aca8c5b74f54945f04094d348057'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:04.088540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8249aca8c5b74f54945f04094d348057'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:04.095358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8249aca8c5b74f54945f04094d348057'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:04.107771 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 943/1883] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:03 2022] DELETE /identity/v3/users/8249aca8c5b74f54945f04094d348057 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:41:04.112171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddb07b-bc55-46ab-9433-4e19b7e48eda None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:04.112691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddb07b-bc55-46ab-9433-4e19b7e48eda None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:04.112885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddb07b-bc55-46ab-9433-4e19b7e48eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:04.135192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c4ddb07b-bc55-46ab-9433-4e19b7e48eda None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:04.137720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c4ddb07b-bc55-46ab-9433-4e19b7e48eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:04.199074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 941/1884] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:04.209378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:04.210921 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:04.271824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcda99a-2f1e-4002-8ed5-129250a2cb62 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:04.276074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f0a3d-9984-4686-ad7e-c29b4d95ce75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:04.276842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f0a3d-9984-4686-ad7e-c29b4d95ce75 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:04.277125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f0a3d-9984-4686-ad7e-c29b4d95ce75 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:04.277403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f0a3d-9984-4686-ad7e-c29b4d95ce75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:04.286632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f0a3d-9984-4686-ad7e-c29b4d95ce75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:04.288052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f0a3d-9984-4686-ad7e-c29b4d95ce75 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:04.300668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 944/1885] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:04.580059 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:04.774784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddb07b-bc55-46ab-9433-4e19b7e48eda None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:04.777573 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c4ddb07b-bc55-46ab-9433-4e19b7e48eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:04.796737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddb07b-bc55-46ab-9433-4e19b7e48eda None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:04.802326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:04.805837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:04.805837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:04.805837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] PATH_INFO: `/v3/projects/c7c8e17399fc42d5839bda667d2ebe75` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:04.805837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7c8e17399fc42d5839bda667d2ebe75)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:04.806843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:04.898281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7c8e17399fc42d5839bda667d2ebe75', 'request_id': 'req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:04.905805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7c8e17399fc42d5839bda667d2ebe75', 'request_id': 'req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:04.976034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7c8e17399fc42d5839bda667d2ebe75', 'request_id': 'req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:04.987160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] The token cache is being invalidate because project c7c8e17399fc42d5839bda667d2ebe75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:04.987449 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:04.988072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:04.989574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 942/1886] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:04 2022] DELETE /identity/v3/projects/c7c8e17399fc42d5839bda667d2ebe75 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:04.993756 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f0a3d-9984-4686-ad7e-c29b4d95ce75 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:05.003586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f0a3d-9984-4686-ad7e-c29b4d95ce75 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:05.059865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25be9a-7cdd-4deb-a1c4-042c319b2765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:05.060501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25be9a-7cdd-4deb-a1c4-042c319b2765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:05.060785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25be9a-7cdd-4deb-a1c4-042c319b2765 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:05.061056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25be9a-7cdd-4deb-a1c4-042c319b2765 admin admin] PATH_INFO: `/v3/users/3398dd480985445fbcd08573d105958d` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:05.062238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25be9a-7cdd-4deb-a1c4-042c319b2765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3398dd480985445fbcd08573d105958d)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:05.064133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25be9a-7cdd-4deb-a1c4-042c319b2765 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:05.160714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:05.241537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3f25be9a-7cdd-4deb-a1c4-042c319b2765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3398dd480985445fbcd08573d105958d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:05.326392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3f25be9a-7cdd-4deb-a1c4-042c319b2765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3398dd480985445fbcd08573d105958d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:05.339669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3f25be9a-7cdd-4deb-a1c4-042c319b2765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3398dd480985445fbcd08573d105958d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:05.340453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3f25be9a-7cdd-4deb-a1c4-042c319b2765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3398dd480985445fbcd08573d105958d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:05.347345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3f25be9a-7cdd-4deb-a1c4-042c319b2765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3398dd480985445fbcd08573d105958d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:05.353217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 945/1887] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:04 2022] DELETE /identity/v3/users/3398dd480985445fbcd08573d105958d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:05.703193 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:05.954640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:05.956427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:05.973390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316016-c5ca-4a9f-8cf8-4e51fad4f541 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:06.036179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:06.036823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:06.037103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:06.037374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] PATH_INFO: `/v3/projects/55613d013a2d4e2f9b30d9d42a16d671` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:06.038562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=55613d013a2d4e2f9b30d9d42a16d671)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:06.040464 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:06.205835 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55613d013a2d4e2f9b30d9d42a16d671', 'request_id': 'req-67f00624-65b8-4e26-8449-1b333b836e0a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:06.212389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55613d013a2d4e2f9b30d9d42a16d671', 'request_id': 'req-67f00624-65b8-4e26-8449-1b333b836e0a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:06.283792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55613d013a2d4e2f9b30d9d42a16d671', 'request_id': 'req-67f00624-65b8-4e26-8449-1b333b836e0a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:06.296330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] The token cache is being invalidate because project 55613d013a2d4e2f9b30d9d42a16d671 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:06.296622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:06.297260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-67f00624-65b8-4e26-8449-1b333b836e0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:06.298850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 943/1888] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:05 2022] DELETE /identity/v3/projects/55613d013a2d4e2f9b30d9d42a16d671 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:06.304162 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428245-5173-4f29-8418-a342f4d6c4b1 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:06.304649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428245-5173-4f29-8418-a342f4d6c4b1 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:06.304755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428245-5173-4f29-8418-a342f4d6c4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:06.324707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ef428245-5173-4f29-8418-a342f4d6c4b1 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:06.325988 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ef428245-5173-4f29-8418-a342f4d6c4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:06.385233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 946/1889] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:06.394885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d936a-26f5-4abb-852d-6aea4e2514df None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:06.397875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d936a-26f5-4abb-852d-6aea4e2514df None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:06.397875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d936a-26f5-4abb-852d-6aea4e2514df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:06.420426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-434d936a-26f5-4abb-852d-6aea4e2514df None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:06.421750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-434d936a-26f5-4abb-852d-6aea4e2514df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:06.429680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 944/1890] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:06.434660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef428245-5173-4f29-8418-a342f4d6c4b1 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:06.444602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef428245-5173-4f29-8418-a342f4d6c4b1 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:06.448962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81b8e1-4e6a-4731-bd1f-30653589e79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:06.449672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81b8e1-4e6a-4731-bd1f-30653589e79b admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:06.449953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81b8e1-4e6a-4731-bd1f-30653589e79b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:06.450271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81b8e1-4e6a-4731-bd1f-30653589e79b admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:06.451034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81b8e1-4e6a-4731-bd1f-30653589e79b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:41:06.451328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81b8e1-4e6a-4731-bd1f-30653589e79b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:06.452905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81b8e1-4e6a-4731-bd1f-30653589e79b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:06.464690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 947/1891] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:41:06.469056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d936a-26f5-4abb-852d-6aea4e2514df None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:06.478273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d936a-26f5-4abb-852d-6aea4e2514df None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:06.483897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548f75a-3ae3-4678-8647-23c4e62b0647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:06.484505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548f75a-3ae3-4678-8647-23c4e62b0647 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:06.484786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548f75a-3ae3-4678-8647-23c4e62b0647 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:06.485060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548f75a-3ae3-4678-8647-23c4e62b0647 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:06.485723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548f75a-3ae3-4678-8647-23c4e62b0647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:06.487649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548f75a-3ae3-4678-8647-23c4e62b0647 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:06.493782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:06.520563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:06.591554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 945/1892] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:41:06 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:06.595859 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81b8e1-4e6a-4731-bd1f-30653589e79b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:06.606929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81b8e1-4e6a-4731-bd1f-30653589e79b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:06.664588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ea864-aa35-43cf-b341-e5704c723ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:06.665222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ea864-aa35-43cf-b341-e5704c723ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:06.665501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ea864-aa35-43cf-b341-e5704c723ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:06.665771 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ea864-aa35-43cf-b341-e5704c723ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:06.667328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ea864-aa35-43cf-b341-e5704c723ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:06.669445 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ea864-aa35-43cf-b341-e5704c723ab0 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:06.782268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 948/1893] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:41:06 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:06.782380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548f75a-3ae3-4678-8647-23c4e62b0647 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:06.794662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548f75a-3ae3-4678-8647-23c4e62b0647 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:06.803119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407289d-b41b-4c39-97e9-12aa455de8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:06.804163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407289d-b41b-4c39-97e9-12aa455de8c3 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:06.804645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407289d-b41b-4c39-97e9-12aa455de8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:06.805135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407289d-b41b-4c39-97e9-12aa455de8c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:06.805930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407289d-b41b-4c39-97e9-12aa455de8c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:06.807762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407289d-b41b-4c39-97e9-12aa455de8c3 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:06.820010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 946/1894] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:06.823961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ea864-aa35-43cf-b341-e5704c723ab0 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:06.834345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ea864-aa35-43cf-b341-e5704c723ab0 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:06.840138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082de45c-ad8a-4da8-bb13-ef35752077ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:06.840770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082de45c-ad8a-4da8-bb13-ef35752077ad admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:06.841048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082de45c-ad8a-4da8-bb13-ef35752077ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:06.841322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082de45c-ad8a-4da8-bb13-ef35752077ad admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:06.842039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082de45c-ad8a-4da8-bb13-ef35752077ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:06.843620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082de45c-ad8a-4da8-bb13-ef35752077ad admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:06.857033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 949/1895] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:06.861482 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407289d-b41b-4c39-97e9-12aa455de8c3 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:06.875643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407289d-b41b-4c39-97e9-12aa455de8c3 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:06.881729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d739e-a025-4308-a36c-7ba83667be71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:06.883062 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d739e-a025-4308-a36c-7ba83667be71 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:06.883350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d739e-a025-4308-a36c-7ba83667be71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:06.883626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d739e-a025-4308-a36c-7ba83667be71 admin admin] PATH_INFO: `/v3/projects/e24351c3caf1443fa08f55a85a5afd72/users/0297931c826647078ee7b25e464643b6/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:06.904202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d739e-a025-4308-a36c-7ba83667be71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24351c3caf1443fa08f55a85a5afd72, user_id=0297931c826647078ee7b25e464643b6, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:06.908975 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d739e-a025-4308-a36c-7ba83667be71 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:06.995596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 947/1896] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:06 2022] PUT /identity/v3/projects/e24351c3caf1443fa08f55a85a5afd72/users/0297931c826647078ee7b25e464643b6/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:07.005927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dff4fe-bd6b-44d3-90da-812a71154aab None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:07.006261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dff4fe-bd6b-44d3-90da-812a71154aab None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:07.006568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dff4fe-bd6b-44d3-90da-812a71154aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:07.021627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:07.037019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-e3dff4fe-bd6b-44d3-90da-812a71154aab None None] MFA Rules not processed for user `0297931c826647078ee7b25e464643b6`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:07.038268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e3dff4fe-bd6b-44d3-90da-812a71154aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:07.051410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:07.103861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 950/1897] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:41:07.108535 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d739e-a025-4308-a36c-7ba83667be71 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:07.173716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d739e-a025-4308-a36c-7ba83667be71 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:07.178439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96afabb-f293-4370-b650-a9d3a365c04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:07.179063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96afabb-f293-4370-b650-a9d3a365c04a admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:07.179348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96afabb-f293-4370-b650-a9d3a365c04a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:07.179626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96afabb-f293-4370-b650-a9d3a365c04a admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:07.180312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96afabb-f293-4370-b650-a9d3a365c04a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:07.183159 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96afabb-f293-4370-b650-a9d3a365c04a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:07.271042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 948/1898] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:41:07 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:41:07.275491 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dff4fe-bd6b-44d3-90da-812a71154aab None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:07.290779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dff4fe-bd6b-44d3-90da-812a71154aab None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:07.349470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d6736-e389-4bd0-ba69-d03b99ed02b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:07.350136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6736-e389-4bd0-ba69-d03b99ed02b2 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:07.350386 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6736-e389-4bd0-ba69-d03b99ed02b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:07.350658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6736-e389-4bd0-ba69-d03b99ed02b2 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:07.351319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d6736-e389-4bd0-ba69-d03b99ed02b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:07.353297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d6736-e389-4bd0-ba69-d03b99ed02b2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:07.362502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:07.480409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 951/1899] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:41:07 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:07.484639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96afabb-f293-4370-b650-a9d3a365c04a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:07.495504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96afabb-f293-4370-b650-a9d3a365c04a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:07.500433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e8637-8655-4392-af32-4d84b090098f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:07.501131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e8637-8655-4392-af32-4d84b090098f admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:07.501422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e8637-8655-4392-af32-4d84b090098f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:07.501909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e8637-8655-4392-af32-4d84b090098f admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:07.502506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e8637-8655-4392-af32-4d84b090098f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:07.504108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e8637-8655-4392-af32-4d84b090098f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:07.515588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 949/1900] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:07.519757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d6736-e389-4bd0-ba69-d03b99ed02b2 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:07.530241 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d6736-e389-4bd0-ba69-d03b99ed02b2 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:07.534998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a778b-1b6e-4a6b-8bbc-62bcc2f3ef6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:07.535695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a778b-1b6e-4a6b-8bbc-62bcc2f3ef6e admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:07.535992 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a778b-1b6e-4a6b-8bbc-62bcc2f3ef6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:07.536294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a778b-1b6e-4a6b-8bbc-62bcc2f3ef6e admin admin] PATH_INFO: `/v3/projects/8b299da5faae412d9f2d93f8c867847b/users/a89d397ba03343f0b5d1b065925c6391/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:07.543801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:07.552093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a778b-1b6e-4a6b-8bbc-62bcc2f3ef6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b299da5faae412d9f2d93f8c867847b, user_id=a89d397ba03343f0b5d1b065925c6391, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:07.554764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a778b-1b6e-4a6b-8bbc-62bcc2f3ef6e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:07.599987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 952/1901] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:07 2022] PUT /identity/v3/projects/8b299da5faae412d9f2d93f8c867847b/users/a89d397ba03343f0b5d1b065925c6391/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:07.604090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e8637-8655-4392-af32-4d84b090098f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:07.614496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e8637-8655-4392-af32-4d84b090098f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:07.635942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:07.671274 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee1652-6e94-4183-8876-6676805cf357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:07.671883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee1652-6e94-4183-8876-6676805cf357 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:07.672162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee1652-6e94-4183-8876-6676805cf357 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:07.672436 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee1652-6e94-4183-8876-6676805cf357 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:07.673129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee1652-6e94-4183-8876-6676805cf357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:07.674824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee1652-6e94-4183-8876-6676805cf357 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:07.685021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 950/1902] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:07.689675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a778b-1b6e-4a6b-8bbc-62bcc2f3ef6e admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:07.700018 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a778b-1b6e-4a6b-8bbc-62bcc2f3ef6e admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:07.704751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d70d66-b628-4682-a152-eb8d78e3dc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:07.705404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d70d66-b628-4682-a152-eb8d78e3dc02 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:07.705680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d70d66-b628-4682-a152-eb8d78e3dc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:07.705952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d70d66-b628-4682-a152-eb8d78e3dc02 admin admin] PATH_INFO: `/v3/projects/8b299da5faae412d9f2d93f8c867847b/users/a89d397ba03343f0b5d1b065925c6391/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:07.707944 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d70d66-b628-4682-a152-eb8d78e3dc02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b299da5faae412d9f2d93f8c867847b, user_id=a89d397ba03343f0b5d1b065925c6391, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:07.710662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d70d66-b628-4682-a152-eb8d78e3dc02 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:07.712920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:07.720966 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 953/1903] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:07 2022] PUT /identity/v3/projects/8b299da5faae412d9f2d93f8c867847b/users/a89d397ba03343f0b5d1b065925c6391/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:07.732761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103cd3a-b86d-49b9-9eb7-7d1f2240414c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:07.733051 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103cd3a-b86d-49b9-9eb7-7d1f2240414c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:07.733328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103cd3a-b86d-49b9-9eb7-7d1f2240414c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:07.763302 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0103cd3a-b86d-49b9-9eb7-7d1f2240414c None None] MFA Rules not processed for user `a89d397ba03343f0b5d1b065925c6391`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:07.764518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0103cd3a-b86d-49b9-9eb7-7d1f2240414c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:07.810207 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:07.834613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 951/1904] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:07.846098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d70d66-b628-4682-a152-eb8d78e3dc02 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:07.847845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e9d70d66-b628-4682-a152-eb8d78e3dc02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:07.892221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:07.909674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d70d66-b628-4682-a152-eb8d78e3dc02 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:07.913451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18614b76-c3ab-4992-a3f6-a6fa5dd90b2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:07.914048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18614b76-c3ab-4992-a3f6-a6fa5dd90b2f service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:07.914254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18614b76-c3ab-4992-a3f6-a6fa5dd90b2f service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:07.914521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18614b76-c3ab-4992-a3f6-a6fa5dd90b2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:07.924299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18614b76-c3ab-4992-a3f6-a6fa5dd90b2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:07.925637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18614b76-c3ab-4992-a3f6-a6fa5dd90b2f service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:07.939432 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 954/1905] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:41:08.064020 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:08.121486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:08.216016 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:08.372654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:08.743830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:08.852435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:08.958838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103cd3a-b86d-49b9-9eb7-7d1f2240414c None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:09.035049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103cd3a-b86d-49b9-9eb7-7d1f2240414c None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:09.045310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:09.046113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:09.046459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:09.046757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] PATH_INFO: `/v3/users/0297931c826647078ee7b25e464643b6` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:09.048058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0297931c826647078ee7b25e464643b6)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:09.050157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:09.155436 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:09.263909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0297931c826647078ee7b25e464643b6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:09.365901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0297931c826647078ee7b25e464643b6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:09.366716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0297931c826647078ee7b25e464643b6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:09.371776 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:09.382423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0297931c826647078ee7b25e464643b6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:09.393796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0297931c826647078ee7b25e464643b6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:09.396382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 952/1906] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:08 2022] DELETE /identity/v3/users/0297931c826647078ee7b25e464643b6 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:09.401465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285a211-ff48-47af-8234-369d7b472a71 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:09.402446 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285a211-ff48-47af-8234-369d7b472a71 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:09.402974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285a211-ff48-47af-8234-369d7b472a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:09.427331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f285a211-ff48-47af-8234-369d7b472a71 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:09.428644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f285a211-ff48-47af-8234-369d7b472a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:09.489030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 955/1907] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:09.499729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:09.501293 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:09.561363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dcb34-75ce-4f1e-922f-3f671272cc31 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:09.565388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74cff0-5b2d-4475-8437-e1916a9ec221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:09.566088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74cff0-5b2d-4475-8437-e1916a9ec221 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:09.566396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74cff0-5b2d-4475-8437-e1916a9ec221 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:09.566668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74cff0-5b2d-4475-8437-e1916a9ec221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:09.578590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74cff0-5b2d-4475-8437-e1916a9ec221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:09.580083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74cff0-5b2d-4475-8437-e1916a9ec221 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:09.592059 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 953/1908] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:09.869990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:10.055093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285a211-ff48-47af-8234-369d7b472a71 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:10.055242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f285a211-ff48-47af-8234-369d7b472a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:10.067801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285a211-ff48-47af-8234-369d7b472a71 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:10.072463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:10.073131 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:10.073415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:10.076743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] PATH_INFO: `/v3/projects/e24351c3caf1443fa08f55a85a5afd72` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:10.078064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24351c3caf1443fa08f55a85a5afd72)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:10.080109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:10.194987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24351c3caf1443fa08f55a85a5afd72', 'request_id': 'req-9469cc2d-c98f-4680-9dc9-c486ddefd490'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:10.199104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:10.261385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24351c3caf1443fa08f55a85a5afd72', 'request_id': 'req-9469cc2d-c98f-4680-9dc9-c486ddefd490'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:10.268644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24351c3caf1443fa08f55a85a5afd72', 'request_id': 'req-9469cc2d-c98f-4680-9dc9-c486ddefd490'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:10.272566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:10.284007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] The token cache is being invalidate because project e24351c3caf1443fa08f55a85a5afd72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:10.284249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:10.285078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:10.287010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 956/1909] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:10 2022] DELETE /identity/v3/projects/e24351c3caf1443fa08f55a85a5afd72 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:10.291072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74cff0-5b2d-4475-8437-e1916a9ec221 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:10.315947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74cff0-5b2d-4475-8437-e1916a9ec221 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:10.408883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae77d0b-2b46-4265-952c-6751727a50c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:10.409612 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae77d0b-2b46-4265-952c-6751727a50c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:10.409964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae77d0b-2b46-4265-952c-6751727a50c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:10.410331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae77d0b-2b46-4265-952c-6751727a50c3 admin admin] PATH_INFO: `/v3/users/a89d397ba03343f0b5d1b065925c6391` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:10.411958 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae77d0b-2b46-4265-952c-6751727a50c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a89d397ba03343f0b5d1b065925c6391)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:10.414253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae77d0b-2b46-4265-952c-6751727a50c3 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:10.711794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-bae77d0b-2b46-4265-952c-6751727a50c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a89d397ba03343f0b5d1b065925c6391'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:10.823907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-bae77d0b-2b46-4265-952c-6751727a50c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a89d397ba03343f0b5d1b065925c6391'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:10.824550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-bae77d0b-2b46-4265-952c-6751727a50c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a89d397ba03343f0b5d1b065925c6391'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:10.852993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-bae77d0b-2b46-4265-952c-6751727a50c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a89d397ba03343f0b5d1b065925c6391'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:10.860872 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:10.863869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-bae77d0b-2b46-4265-952c-6751727a50c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a89d397ba03343f0b5d1b065925c6391'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:10.872832 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 954/1910] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:10 2022] DELETE /identity/v3/users/a89d397ba03343f0b5d1b065925c6391 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:11.369878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:11.371658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:11.396760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469cc2d-c98f-4680-9dc9-c486ddefd490 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:11.448781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:11.450986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:11.450986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:11.450986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] PATH_INFO: `/v3/projects/8b299da5faae412d9f2d93f8c867847b` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:11.451235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b299da5faae412d9f2d93f8c867847b)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:11.453014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:11.584371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b299da5faae412d9f2d93f8c867847b', 'request_id': 'req-60175223-8f25-49d7-a230-9304a98c1ebd'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:11.652706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b299da5faae412d9f2d93f8c867847b', 'request_id': 'req-60175223-8f25-49d7-a230-9304a98c1ebd'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:11.659334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b299da5faae412d9f2d93f8c867847b', 'request_id': 'req-60175223-8f25-49d7-a230-9304a98c1ebd'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:11.674718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] The token cache is being invalidate because project 8b299da5faae412d9f2d93f8c867847b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:11.675017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:11.675691 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-60175223-8f25-49d7-a230-9304a98c1ebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:11.677421 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 957/1911] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:11 2022] DELETE /identity/v3/projects/8b299da5faae412d9f2d93f8c867847b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:11.684488 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de462c4-ebf2-475d-8fbc-b4ca20be9cdf None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:11.685050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de462c4-ebf2-475d-8fbc-b4ca20be9cdf None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:11.685455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de462c4-ebf2-475d-8fbc-b4ca20be9cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:11.711319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0de462c4-ebf2-475d-8fbc-b4ca20be9cdf None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:11.712753 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0de462c4-ebf2-475d-8fbc-b4ca20be9cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:11.782354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 955/1912] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:11.792796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c4805-3072-4e26-b5b5-41aee8532c59 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:11.793088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c4805-3072-4e26-b5b5-41aee8532c59 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:11.793368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c4805-3072-4e26-b5b5-41aee8532c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:11.813350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bc7c4805-3072-4e26-b5b5-41aee8532c59 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:11.814577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bc7c4805-3072-4e26-b5b5-41aee8532c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:11.823452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 958/1913] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:11.827626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de462c4-ebf2-475d-8fbc-b4ca20be9cdf None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:11.837238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de462c4-ebf2-475d-8fbc-b4ca20be9cdf None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:11.842173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b51301-5fa0-48f6-8e12-ebb697f5a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:11.842783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b51301-5fa0-48f6-8e12-ebb697f5a30d admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:11.843070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b51301-5fa0-48f6-8e12-ebb697f5a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:11.843350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b51301-5fa0-48f6-8e12-ebb697f5a30d admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:11.844049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b51301-5fa0-48f6-8e12-ebb697f5a30d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:41:11.844358 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b51301-5fa0-48f6-8e12-ebb697f5a30d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:11.845959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b51301-5fa0-48f6-8e12-ebb697f5a30d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:11.857590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 956/1914] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:41:11.861838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c4805-3072-4e26-b5b5-41aee8532c59 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:11.871974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c4805-3072-4e26-b5b5-41aee8532c59 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:11.876951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da77e2-0579-49b7-b5fb-178ebf431f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:11.877333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da77e2-0579-49b7-b5fb-178ebf431f7c admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:11.877617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da77e2-0579-49b7-b5fb-178ebf431f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:11.877893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da77e2-0579-49b7-b5fb-178ebf431f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:11.878624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da77e2-0579-49b7-b5fb-178ebf431f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:11.880530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da77e2-0579-49b7-b5fb-178ebf431f7c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:11.971760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 959/1915] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:41:11 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:41:11.972914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b51301-5fa0-48f6-8e12-ebb697f5a30d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:11.984532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b51301-5fa0-48f6-8e12-ebb697f5a30d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:12.045696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bad846-35c9-4d9d-b010-971c77b0f43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:12.046346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bad846-35c9-4d9d-b010-971c77b0f43e admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:12.046627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bad846-35c9-4d9d-b010-971c77b0f43e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:12.046898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bad846-35c9-4d9d-b010-971c77b0f43e admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:12.047558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bad846-35c9-4d9d-b010-971c77b0f43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:12.049471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bad846-35c9-4d9d-b010-971c77b0f43e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:12.126622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 957/1916] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:41:11 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:41:12.130816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da77e2-0579-49b7-b5fb-178ebf431f7c admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:12.136981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:12.142280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da77e2-0579-49b7-b5fb-178ebf431f7c admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:12.147002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079232dd-000a-4bc9-a197-03ace801afd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:12.147880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079232dd-000a-4bc9-a197-03ace801afd4 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:12.148165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079232dd-000a-4bc9-a197-03ace801afd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:12.148440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079232dd-000a-4bc9-a197-03ace801afd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:12.149126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079232dd-000a-4bc9-a197-03ace801afd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:12.151024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079232dd-000a-4bc9-a197-03ace801afd4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:12.160772 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 960/1917] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:12.164720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bad846-35c9-4d9d-b010-971c77b0f43e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:12.176750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bad846-35c9-4d9d-b010-971c77b0f43e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:12.181990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf62c8b-871d-4c19-9a68-25b385a89b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:12.183473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf62c8b-871d-4c19-9a68-25b385a89b51 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:12.183773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf62c8b-871d-4c19-9a68-25b385a89b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:12.184044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf62c8b-871d-4c19-9a68-25b385a89b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:12.184738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf62c8b-871d-4c19-9a68-25b385a89b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:12.186432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf62c8b-871d-4c19-9a68-25b385a89b51 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:12.197716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 958/1918] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:12.202566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079232dd-000a-4bc9-a197-03ace801afd4 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:12.212669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079232dd-000a-4bc9-a197-03ace801afd4 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:12.217851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ebe79-2e6d-43d9-acbf-8e0b72d44d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:12.218557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ebe79-2e6d-43d9-acbf-8e0b72d44d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:12.218836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ebe79-2e6d-43d9-acbf-8e0b72d44d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:12.219108 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ebe79-2e6d-43d9-acbf-8e0b72d44d32 admin admin] PATH_INFO: `/v3/projects/d292dbbd22264d1386204244cb7babdb/users/25b48266714b4fe585701b6ba41a796b/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:12.235464 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ebe79-2e6d-43d9-acbf-8e0b72d44d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d292dbbd22264d1386204244cb7babdb, user_id=25b48266714b4fe585701b6ba41a796b, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:12.238112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ebe79-2e6d-43d9-acbf-8e0b72d44d32 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:12.321535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 961/1919] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:12 2022] PUT /identity/v3/projects/d292dbbd22264d1386204244cb7babdb/users/25b48266714b4fe585701b6ba41a796b/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:41:12.332335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376afe7-6f33-4409-8656-789283f19794 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:12.332628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376afe7-6f33-4409-8656-789283f19794 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:12.332909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376afe7-6f33-4409-8656-789283f19794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:12.361175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0376afe7-6f33-4409-8656-789283f19794 None None] MFA Rules not processed for user `25b48266714b4fe585701b6ba41a796b`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:12.362506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0376afe7-6f33-4409-8656-789283f19794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:12.412162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:12.430221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 959/1920] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:12.435624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ebe79-2e6d-43d9-acbf-8e0b72d44d32 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:12.499551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ebe79-2e6d-43d9-acbf-8e0b72d44d32 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:12.504803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249fad1-a255-469d-9097-96158c082b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:12.505428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249fad1-a255-469d-9097-96158c082b88 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:12.505714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249fad1-a255-469d-9097-96158c082b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:12.505992 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249fad1-a255-469d-9097-96158c082b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:12.506818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249fad1-a255-469d-9097-96158c082b88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:12.508677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249fad1-a255-469d-9097-96158c082b88 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:12.638318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 962/1921] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:41:12 2022] POST /identity/v3/projects => generated 385 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:12.642682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376afe7-6f33-4409-8656-789283f19794 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:12.653637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376afe7-6f33-4409-8656-789283f19794 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:12.713526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61017be0-fcd9-4c05-9c27-5eec14dddd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:12.714241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61017be0-fcd9-4c05-9c27-5eec14dddd5c admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:12.714494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61017be0-fcd9-4c05-9c27-5eec14dddd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:12.714888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61017be0-fcd9-4c05-9c27-5eec14dddd5c admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:12.715570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61017be0-fcd9-4c05-9c27-5eec14dddd5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:12.717561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61017be0-fcd9-4c05-9c27-5eec14dddd5c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:12.824386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 960/1922] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:41:12 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:41:12.828543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249fad1-a255-469d-9097-96158c082b88 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:12.839842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249fad1-a255-469d-9097-96158c082b88 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:12.847610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06f7ce-e684-4623-aac1-9f066b3ce9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:12.848277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06f7ce-e684-4623-aac1-9f066b3ce9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:12.848568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06f7ce-e684-4623-aac1-9f066b3ce9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:12.848854 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06f7ce-e684-4623-aac1-9f066b3ce9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:12.849620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06f7ce-e684-4623-aac1-9f066b3ce9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:12.851424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06f7ce-e684-4623-aac1-9f066b3ce9d9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:12.863887 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 963/1923] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:41:12.868299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61017be0-fcd9-4c05-9c27-5eec14dddd5c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:12.880306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61017be0-fcd9-4c05-9c27-5eec14dddd5c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:12.889905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb1a28-7954-4ec0-a06d-2206fa32b3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:12.890567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb1a28-7954-4ec0-a06d-2206fa32b3e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:12.890849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb1a28-7954-4ec0-a06d-2206fa32b3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:12.891122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb1a28-7954-4ec0-a06d-2206fa32b3e5 admin admin] PATH_INFO: `/v3/projects/c85f9fa7537d4904bbea0c0e57e95bdd/users/b00006b65cc04966b130ff8b97291ca9/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:12.910740 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb1a28-7954-4ec0-a06d-2206fa32b3e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85f9fa7537d4904bbea0c0e57e95bdd, user_id=b00006b65cc04966b130ff8b97291ca9, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:12.913383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb1a28-7954-4ec0-a06d-2206fa32b3e5 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:12.983701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 961/1924] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:12 2022] PUT /identity/v3/projects/c85f9fa7537d4904bbea0c0e57e95bdd/users/b00006b65cc04966b130ff8b97291ca9/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:12.989605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06f7ce-e684-4623-aac1-9f066b3ce9d9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:12.999911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06f7ce-e684-4623-aac1-9f066b3ce9d9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:13.055487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89ead8-3c5d-4bbd-bf48-bf8b38dc7298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:13.056113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89ead8-3c5d-4bbd-bf48-bf8b38dc7298 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:13.056396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89ead8-3c5d-4bbd-bf48-bf8b38dc7298 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:13.056673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89ead8-3c5d-4bbd-bf48-bf8b38dc7298 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:13.057369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89ead8-3c5d-4bbd-bf48-bf8b38dc7298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:13.059132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89ead8-3c5d-4bbd-bf48-bf8b38dc7298 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:13.068812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 964/1925] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:13.074036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb1a28-7954-4ec0-a06d-2206fa32b3e5 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:13.092706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb1a28-7954-4ec0-a06d-2206fa32b3e5 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:13.097320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2338ab-4b1c-4acc-81f8-af9fcb2aec87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:13.097983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2338ab-4b1c-4acc-81f8-af9fcb2aec87 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:13.098296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2338ab-4b1c-4acc-81f8-af9fcb2aec87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:13.098567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2338ab-4b1c-4acc-81f8-af9fcb2aec87 admin admin] PATH_INFO: `/v3/projects/c85f9fa7537d4904bbea0c0e57e95bdd/users/b00006b65cc04966b130ff8b97291ca9/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:13.100494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2338ab-4b1c-4acc-81f8-af9fcb2aec87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85f9fa7537d4904bbea0c0e57e95bdd, user_id=b00006b65cc04966b130ff8b97291ca9, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:13.103150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2338ab-4b1c-4acc-81f8-af9fcb2aec87 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:13.114330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 962/1926] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:13 2022] PUT /identity/v3/projects/c85f9fa7537d4904bbea0c0e57e95bdd/users/b00006b65cc04966b130ff8b97291ca9/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:13.126456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b629300-add0-492d-ba82-f61299bd180e None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:13.126748 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b629300-add0-492d-ba82-f61299bd180e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:13.127026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b629300-add0-492d-ba82-f61299bd180e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:13.156698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-5b629300-add0-492d-ba82-f61299bd180e None None] MFA Rules not processed for user `b00006b65cc04966b130ff8b97291ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:13.157932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5b629300-add0-492d-ba82-f61299bd180e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:13.187527 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:13.230910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 965/1927] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:13.243934 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2338ab-4b1c-4acc-81f8-af9fcb2aec87 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:13.246278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6e2338ab-4b1c-4acc-81f8-af9fcb2aec87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:13.310976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2338ab-4b1c-4acc-81f8-af9fcb2aec87 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:13.314623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195a2f2-3e8c-4f9e-be78-ad05febef85f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:13.315269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195a2f2-3e8c-4f9e-be78-ad05febef85f service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:13.315553 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195a2f2-3e8c-4f9e-be78-ad05febef85f service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:13.315827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195a2f2-3e8c-4f9e-be78-ad05febef85f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:13.325554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195a2f2-3e8c-4f9e-be78-ad05febef85f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:13.327018 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195a2f2-3e8c-4f9e-be78-ad05febef85f service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:13.336304 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 963/1928] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:14.470540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b629300-add0-492d-ba82-f61299bd180e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:14.533669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b629300-add0-492d-ba82-f61299bd180e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:14.538273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:14.538917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:14.539201 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:14.539476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] PATH_INFO: `/v3/users/25b48266714b4fe585701b6ba41a796b` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:14.540623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b48266714b4fe585701b6ba41a796b)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:14.542667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:14.745248 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b48266714b4fe585701b6ba41a796b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:14.828983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b48266714b4fe585701b6ba41a796b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:14.841970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b48266714b4fe585701b6ba41a796b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:14.842657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b48266714b4fe585701b6ba41a796b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:14.849632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b48266714b4fe585701b6ba41a796b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:14.855705 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 966/1929] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:14 2022] DELETE /identity/v3/users/25b48266714b4fe585701b6ba41a796b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:14.860472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a4fea-435c-41de-92c3-68f1efa02c2f None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:14.860926 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a4fea-435c-41de-92c3-68f1efa02c2f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:14.861039 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a4fea-435c-41de-92c3-68f1efa02c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:14.880785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-cd2a4fea-435c-41de-92c3-68f1efa02c2f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:14.881977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-cd2a4fea-435c-41de-92c3-68f1efa02c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:14.922935 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:14.939970 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 964/1930] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:41:14.950868 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:14.952330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:14.999290 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:15.010697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7855b50-d718-4ec5-b54d-c23b84d6528c admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:15.019494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613bbd5-f2e6-4513-91bd-898f98d7f910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:15.019494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613bbd5-f2e6-4513-91bd-898f98d7f910 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:15.019494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613bbd5-f2e6-4513-91bd-898f98d7f910 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:15.019494 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613bbd5-f2e6-4513-91bd-898f98d7f910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:15.029015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613bbd5-f2e6-4513-91bd-898f98d7f910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:15.030498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613bbd5-f2e6-4513-91bd-898f98d7f910 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:15.045335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 967/1931] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:15.551089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a4fea-435c-41de-92c3-68f1efa02c2f None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:15.552821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-cd2a4fea-435c-41de-92c3-68f1efa02c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:15.566174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a4fea-435c-41de-92c3-68f1efa02c2f None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:15.571013 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:15.571807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:15.572383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:15.572719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] PATH_INFO: `/v3/projects/d292dbbd22264d1386204244cb7babdb` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:15.574687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d292dbbd22264d1386204244cb7babdb)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:15.576632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:15.692921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd292dbbd22264d1386204244cb7babdb', 'request_id': 'req-2589048a-ed52-44dc-92ed-57f98b44c792'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:15.699662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd292dbbd22264d1386204244cb7babdb', 'request_id': 'req-2589048a-ed52-44dc-92ed-57f98b44c792'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:15.809543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd292dbbd22264d1386204244cb7babdb', 'request_id': 'req-2589048a-ed52-44dc-92ed-57f98b44c792'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:15.820299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] The token cache is being invalidate because project d292dbbd22264d1386204244cb7babdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:15.820584 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:15.821195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:15.826605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 965/1932] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:15 2022] DELETE /identity/v3/projects/d292dbbd22264d1386204244cb7babdb => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:15.827562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613bbd5-f2e6-4513-91bd-898f98d7f910 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:15.837794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613bbd5-f2e6-4513-91bd-898f98d7f910 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:15.894785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e79f44-8484-43b5-b48e-e2551ade241c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:15.895423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e79f44-8484-43b5-b48e-e2551ade241c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:15.895711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e79f44-8484-43b5-b48e-e2551ade241c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:15.895990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e79f44-8484-43b5-b48e-e2551ade241c admin admin] PATH_INFO: `/v3/users/b00006b65cc04966b130ff8b97291ca9` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:15.897170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e79f44-8484-43b5-b48e-e2551ade241c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b00006b65cc04966b130ff8b97291ca9)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:15.899115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e79f44-8484-43b5-b48e-e2551ade241c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:16.082219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b1e79f44-8484-43b5-b48e-e2551ade241c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b00006b65cc04966b130ff8b97291ca9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:16.135542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b1e79f44-8484-43b5-b48e-e2551ade241c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b00006b65cc04966b130ff8b97291ca9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:16.148123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b1e79f44-8484-43b5-b48e-e2551ade241c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b00006b65cc04966b130ff8b97291ca9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:16.148958 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b1e79f44-8484-43b5-b48e-e2551ade241c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b00006b65cc04966b130ff8b97291ca9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:16.155934 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b1e79f44-8484-43b5-b48e-e2551ade241c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b00006b65cc04966b130ff8b97291ca9'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:16.162181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 968/1933] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:15 2022] DELETE /identity/v3/users/b00006b65cc04966b130ff8b97291ca9 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:16.643836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:16.646837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:16.660333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589048a-ed52-44dc-92ed-57f98b44c792 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:16.727787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:16.728437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:16.728858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:16.729284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] PATH_INFO: `/v3/projects/c85f9fa7537d4904bbea0c0e57e95bdd` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:16.730756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c85f9fa7537d4904bbea0c0e57e95bdd)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:16.732713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:16.846732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c85f9fa7537d4904bbea0c0e57e95bdd', 'request_id': 'req-e05f5144-e282-4d03-b8e0-218020e2ed85'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:16.852787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c85f9fa7537d4904bbea0c0e57e95bdd', 'request_id': 'req-e05f5144-e282-4d03-b8e0-218020e2ed85'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:16.919849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c85f9fa7537d4904bbea0c0e57e95bdd', 'request_id': 'req-e05f5144-e282-4d03-b8e0-218020e2ed85'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:16.927571 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:16.932072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] The token cache is being invalidate because project c85f9fa7537d4904bbea0c0e57e95bdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:16.933179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:16.933840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e05f5144-e282-4d03-b8e0-218020e2ed85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:16.935484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 966/1934] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:16 2022] DELETE /identity/v3/projects/c85f9fa7537d4904bbea0c0e57e95bdd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:16.942309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd332c-a13b-443e-9552-c64b218ec5ce None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:16.942791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd332c-a13b-443e-9552-c64b218ec5ce None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:16.942899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd332c-a13b-443e-9552-c64b218ec5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:16.964273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-d4bd332c-a13b-443e-9552-c64b218ec5ce None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:16.965526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d4bd332c-a13b-443e-9552-c64b218ec5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:17.024628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 969/1935] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:17.032922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02b2c-a510-42f0-ada8-cb082b2cad7d None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:17.033496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02b2c-a510-42f0-ada8-cb082b2cad7d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:17.033957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02b2c-a510-42f0-ada8-cb082b2cad7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:17.054923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1de02b2c-a510-42f0-ada8-cb082b2cad7d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:17.056176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1de02b2c-a510-42f0-ada8-cb082b2cad7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:17.064483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 967/1936] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:41:17.069070 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd332c-a13b-443e-9552-c64b218ec5ce None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:17.075404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:17.079491 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd332c-a13b-443e-9552-c64b218ec5ce None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:17.084430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fd03d-d817-472e-892e-05b2318be59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:17.085045 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fd03d-d817-472e-892e-05b2318be59a admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:17.085325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fd03d-d817-472e-892e-05b2318be59a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:17.085594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fd03d-d817-472e-892e-05b2318be59a admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:17.086317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fd03d-d817-472e-892e-05b2318be59a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:41:17.086622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fd03d-d817-472e-892e-05b2318be59a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:17.088785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fd03d-d817-472e-892e-05b2318be59a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:17.105022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 970/1937] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:41:17.106596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de02b2c-a510-42f0-ada8-cb082b2cad7d None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:17.116457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de02b2c-a510-42f0-ada8-cb082b2cad7d None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:17.121537 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c3087-712f-4d74-b768-ccb2b0a742b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:17.127456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c3087-712f-4d74-b768-ccb2b0a742b2 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:17.127456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c3087-712f-4d74-b768-ccb2b0a742b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:17.127456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c3087-712f-4d74-b768-ccb2b0a742b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:17.127456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c3087-712f-4d74-b768-ccb2b0a742b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:17.127456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c3087-712f-4d74-b768-ccb2b0a742b2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:17.260252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 968/1938] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:41:17 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:17.265042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fd03d-d817-472e-892e-05b2318be59a admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:17.275063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fd03d-d817-472e-892e-05b2318be59a admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:17.335920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131ec06-ef39-4c46-9f39-5ffe72e823e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:17.336548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131ec06-ef39-4c46-9f39-5ffe72e823e8 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:17.336835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131ec06-ef39-4c46-9f39-5ffe72e823e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:17.337117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131ec06-ef39-4c46-9f39-5ffe72e823e8 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:17.337819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131ec06-ef39-4c46-9f39-5ffe72e823e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:17.342545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131ec06-ef39-4c46-9f39-5ffe72e823e8 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:17.458267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 971/1939] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:41:17 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:41:17.462390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c3087-712f-4d74-b768-ccb2b0a742b2 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:17.473406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c3087-712f-4d74-b768-ccb2b0a742b2 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:17.478236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9c42d-0325-42e4-aea3-cd7c0badd2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:17.480338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c42d-0325-42e4-aea3-cd7c0badd2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:17.480664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c42d-0325-42e4-aea3-cd7c0badd2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:17.480943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c42d-0325-42e4-aea3-cd7c0badd2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:17.481702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9c42d-0325-42e4-aea3-cd7c0badd2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:17.483357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9c42d-0325-42e4-aea3-cd7c0badd2d8 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:17.499252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 969/1940] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:17.500039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131ec06-ef39-4c46-9f39-5ffe72e823e8 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:17.515206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131ec06-ef39-4c46-9f39-5ffe72e823e8 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:17.531012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9783c9-fe20-4ab3-82ee-4f3d2afae54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:17.531540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9783c9-fe20-4ab3-82ee-4f3d2afae54e admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:17.531822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9783c9-fe20-4ab3-82ee-4f3d2afae54e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:17.532105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9783c9-fe20-4ab3-82ee-4f3d2afae54e admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:17.534221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9783c9-fe20-4ab3-82ee-4f3d2afae54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:17.539277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9783c9-fe20-4ab3-82ee-4f3d2afae54e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:17.558621 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9c42d-0325-42e4-aea3-cd7c0badd2d8 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:17.559385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 972/1941] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:17.571639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9c42d-0325-42e4-aea3-cd7c0badd2d8 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:17.584913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dc9b7-3e1a-44ba-9224-31ab68b5dbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:17.585704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc9b7-3e1a-44ba-9224-31ab68b5dbf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:17.586668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc9b7-3e1a-44ba-9224-31ab68b5dbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:17.586668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc9b7-3e1a-44ba-9224-31ab68b5dbf0 admin admin] PATH_INFO: `/v3/projects/fd9047156c9d49c2a5e1280cc28d9a16/users/adb73f91cf1d482e8c43588ad13bedae/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:17.603231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dc9b7-3e1a-44ba-9224-31ab68b5dbf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9047156c9d49c2a5e1280cc28d9a16, user_id=adb73f91cf1d482e8c43588ad13bedae, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:17.606696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dc9b7-3e1a-44ba-9224-31ab68b5dbf0 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:17.701061 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 970/1942] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:17 2022] PUT /identity/v3/projects/fd9047156c9d49c2a5e1280cc28d9a16/users/adb73f91cf1d482e8c43588ad13bedae/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:17.720388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde10a0-a0cb-4a3c-9c50-0a2e57f1acfd None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:17.720388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde10a0-a0cb-4a3c-9c50-0a2e57f1acfd None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:17.720700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde10a0-a0cb-4a3c-9c50-0a2e57f1acfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:17.755134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-7bde10a0-a0cb-4a3c-9c50-0a2e57f1acfd None None] MFA Rules not processed for user `adb73f91cf1d482e8c43588ad13bedae`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:17.756345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7bde10a0-a0cb-4a3c-9c50-0a2e57f1acfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:17.830595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 973/1943] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:17.842505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dc9b7-3e1a-44ba-9224-31ab68b5dbf0 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:17.844249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9c6dc9b7-3e1a-44ba-9224-31ab68b5dbf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:17.881693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde10a0-a0cb-4a3c-9c50-0a2e57f1acfd None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:17.883116 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7bde10a0-a0cb-4a3c-9c50-0a2e57f1acfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:17.951722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dc9b7-3e1a-44ba-9224-31ab68b5dbf0 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:17.955967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405eb255-0f70-45a5-bc2d-19bdd009609a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a9a6bcfa1b469298774e09a8912b93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:17.956814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405eb255-0f70-45a5-bc2d-19bdd009609a service glance] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:17.957306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405eb255-0f70-45a5-bc2d-19bdd009609a service glance] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:17.957586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405eb255-0f70-45a5-bc2d-19bdd009609a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:17.960805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde10a0-a0cb-4a3c-9c50-0a2e57f1acfd None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:17.966931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:17.967613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:17.967805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405eb255-0f70-45a5-bc2d-19bdd009609a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:17.967956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:17.968414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] PATH_INFO: `/v3/users/b2282f1ab34a472fa918e0f014dfe46d` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:17.969238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405eb255-0f70-45a5-bc2d-19bdd009609a service glance] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:17.969851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2282f1ab34a472fa918e0f014dfe46d)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:17.972059 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:17.983393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:17.983519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 971/1944] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:18.145988 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2282f1ab34a472fa918e0f014dfe46d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:18.238132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2282f1ab34a472fa918e0f014dfe46d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:18.261829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2282f1ab34a472fa918e0f014dfe46d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:18.263002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2282f1ab34a472fa918e0f014dfe46d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:18.277031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2282f1ab34a472fa918e0f014dfe46d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:18.281187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 974/1945] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:17 2022] DELETE /identity/v3/users/b2282f1ab34a472fa918e0f014dfe46d => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:18.285465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2f94-1377-449b-b956-5d9198a3c528 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:18.285756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2f94-1377-449b-b956-5d9198a3c528 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:18.286060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2f94-1377-449b-b956-5d9198a3c528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:18.314651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ad3d2f94-1377-449b-b956-5d9198a3c528 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:18.315995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ad3d2f94-1377-449b-b956-5d9198a3c528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:18.402848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 972/1946] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:18.422145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:18.424483 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:18.536753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162eb6-daad-48fa-825a-b69c5ec1f782 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:18.540940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7316e-0ded-4f39-89ea-1f94d5c80277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:18.541760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7316e-0ded-4f39-89ea-1f94d5c80277 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:18.541971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7316e-0ded-4f39-89ea-1f94d5c80277 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:18.542282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7316e-0ded-4f39-89ea-1f94d5c80277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:18.552183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7316e-0ded-4f39-89ea-1f94d5c80277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:18.553615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7316e-0ded-4f39-89ea-1f94d5c80277 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:18.564517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 975/1947] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:18.812716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1411248-feee-4581-a69f-57830bbbe003 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:18.813275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1411248-feee-4581-a69f-57830bbbe003 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:18.813275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1411248-feee-4581-a69f-57830bbbe003 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:18.814771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 973/1948] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:41:18.849783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e4d58-48f8-4745-9638-8625133771a8 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:18.852577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e4d58-48f8-4745-9638-8625133771a8 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:18.852577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e4d58-48f8-4745-9638-8625133771a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:18.878572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-e50e4d58-48f8-4745-9638-8625133771a8 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:18.881336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e50e4d58-48f8-4745-9638-8625133771a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:19.044886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 976/1949] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:41:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:19.416310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1411248-feee-4581-a69f-57830bbbe003 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:19.416310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d1411248-feee-4581-a69f-57830bbbe003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:19.453694 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1411248-feee-4581-a69f-57830bbbe003 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:19.462748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3f7dc-5a60-48dd-8fd9-84333debc37c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:19.464781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3f7dc-5a60-48dd-8fd9-84333debc37c service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:19.465085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3f7dc-5a60-48dd-8fd9-84333debc37c service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:19.468899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3f7dc-5a60-48dd-8fd9-84333debc37c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:19.475842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3f7dc-5a60-48dd-8fd9-84333debc37c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:19.477455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3f7dc-5a60-48dd-8fd9-84333debc37c service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:19.501233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 974/1950] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:19.631014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e4d58-48f8-4745-9638-8625133771a8 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:19.632635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e50e4d58-48f8-4745-9638-8625133771a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:19.651735 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e4d58-48f8-4745-9638-8625133771a8 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:19.658394 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:19.659122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:19.659411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:19.659807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] PATH_INFO: `/v3/projects/b3256a49c3524e4e9916d8170460a341` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:19.661081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3256a49c3524e4e9916d8170460a341)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:19.665702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:19.675098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:19.787770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3256a49c3524e4e9916d8170460a341', 'request_id': 'req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:19.891234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3256a49c3524e4e9916d8170460a341', 'request_id': 'req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:19.899489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3256a49c3524e4e9916d8170460a341', 'request_id': 'req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:19.915836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] The token cache is being invalidate because project b3256a49c3524e4e9916d8170460a341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:19.916896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:19.917668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:19.919384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 977/1951] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:19 2022] DELETE /identity/v3/projects/b3256a49c3524e4e9916d8170460a341 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:41:19.923327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3f7dc-5a60-48dd-8fd9-84333debc37c service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:19.936956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3f7dc-5a60-48dd-8fd9-84333debc37c service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:19.940996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:20.005457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ea737-8ab1-4ae4-975b-8e8ec7028d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:20.006135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ea737-8ab1-4ae4-975b-8e8ec7028d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:20.006419 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ea737-8ab1-4ae4-975b-8e8ec7028d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:20.006695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ea737-8ab1-4ae4-975b-8e8ec7028d28 admin admin] PATH_INFO: `/v3/users/e77906cdf9dd456eac480c66fc2e267f` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:20.007850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ea737-8ab1-4ae4-975b-8e8ec7028d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77906cdf9dd456eac480c66fc2e267f)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:20.009764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ea737-8ab1-4ae4-975b-8e8ec7028d28 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:20.278786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-127ea737-8ab1-4ae4-975b-8e8ec7028d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77906cdf9dd456eac480c66fc2e267f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:20.343248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-127ea737-8ab1-4ae4-975b-8e8ec7028d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77906cdf9dd456eac480c66fc2e267f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:20.344200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-127ea737-8ab1-4ae4-975b-8e8ec7028d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77906cdf9dd456eac480c66fc2e267f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:20.356845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-127ea737-8ab1-4ae4-975b-8e8ec7028d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77906cdf9dd456eac480c66fc2e267f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:20.363602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-127ea737-8ab1-4ae4-975b-8e8ec7028d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77906cdf9dd456eac480c66fc2e267f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:20.370701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 975/1952] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:19 2022] DELETE /identity/v3/users/e77906cdf9dd456eac480c66fc2e267f => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:20.872360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:20.873868 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:20.899254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd271f2-78e6-4a9d-8f1e-e6f053a3833c admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:20.971304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:20.971896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:20.972239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:20.972517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] PATH_INFO: `/v3/projects/e3e10996d343468093654cf19471181d` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:20.973764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e10996d343468093654cf19471181d)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:20.975729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:21.086906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e10996d343468093654cf19471181d', 'request_id': 'req-8323b43a-1aa8-49e6-951b-892ff4447de7'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:21.188046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e10996d343468093654cf19471181d', 'request_id': 'req-8323b43a-1aa8-49e6-951b-892ff4447de7'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:21.194820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e10996d343468093654cf19471181d', 'request_id': 'req-8323b43a-1aa8-49e6-951b-892ff4447de7'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:21.211341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] The token cache is being invalidate because project e3e10996d343468093654cf19471181d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:21.211739 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:21.212480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8323b43a-1aa8-49e6-951b-892ff4447de7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:21.214554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 978/1953] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:20 2022] DELETE /identity/v3/projects/e3e10996d343468093654cf19471181d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:21.219332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c093d-d737-45b7-9141-d6a7e0c23378 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:21.219911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c093d-d737-45b7-9141-d6a7e0c23378 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:21.219911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c093d-d737-45b7-9141-d6a7e0c23378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:21.240681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-408c093d-d737-45b7-9141-d6a7e0c23378 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:21.241902 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-408c093d-d737-45b7-9141-d6a7e0c23378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:21.308889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 976/1954] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:21.316534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fab635-e517-4e3c-ad80-2279db7a8b69 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:21.316823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fab635-e517-4e3c-ad80-2279db7a8b69 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:21.317100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fab635-e517-4e3c-ad80-2279db7a8b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:21.342698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-d3fab635-e517-4e3c-ad80-2279db7a8b69 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:21.344289 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d3fab635-e517-4e3c-ad80-2279db7a8b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:21.357919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 979/1955] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:21.360229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c093d-d737-45b7-9141-d6a7e0c23378 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:21.375183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c093d-d737-45b7-9141-d6a7e0c23378 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:21.393466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7559ab-ec35-497c-9b51-5016657da536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:21.393466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7559ab-ec35-497c-9b51-5016657da536 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:21.393466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7559ab-ec35-497c-9b51-5016657da536 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:21.393466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7559ab-ec35-497c-9b51-5016657da536 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:21.393777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7559ab-ec35-497c-9b51-5016657da536 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:41:21.394272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7559ab-ec35-497c-9b51-5016657da536 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:21.395826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7559ab-ec35-497c-9b51-5016657da536 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:21.417634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 977/1956] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:41:21.421446 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fab635-e517-4e3c-ad80-2279db7a8b69 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:21.437562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fab635-e517-4e3c-ad80-2279db7a8b69 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:21.457799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f9db7-aeb5-47a2-9334-1506de14c7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:21.458563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f9db7-aeb5-47a2-9334-1506de14c7fb admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:21.458911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f9db7-aeb5-47a2-9334-1506de14c7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:21.459699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f9db7-aeb5-47a2-9334-1506de14c7fb admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:21.460484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f9db7-aeb5-47a2-9334-1506de14c7fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:21.463437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f9db7-aeb5-47a2-9334-1506de14c7fb admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:21.579297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:21.587566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 980/1957] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:41:21 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:41:21.602038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7559ab-ec35-497c-9b51-5016657da536 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:21.611421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7559ab-ec35-497c-9b51-5016657da536 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:21.631063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:21.692183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e2a36-36fd-4d93-8510-adbfd7a9b4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:21.692995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e2a36-36fd-4d93-8510-adbfd7a9b4ed admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:21.693375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e2a36-36fd-4d93-8510-adbfd7a9b4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:21.693721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e2a36-36fd-4d93-8510-adbfd7a9b4ed admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:21.694521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e2a36-36fd-4d93-8510-adbfd7a9b4ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:21.696517 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e2a36-36fd-4d93-8510-adbfd7a9b4ed admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:21.807954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 978/1958] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:41:21 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:21.812154 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f9db7-aeb5-47a2-9334-1506de14c7fb admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:21.823882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f9db7-aeb5-47a2-9334-1506de14c7fb admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:21.828895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbd2e8-2540-4785-a41b-a1b1f0cf6b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:21.829561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbd2e8-2540-4785-a41b-a1b1f0cf6b68 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:21.829847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbd2e8-2540-4785-a41b-a1b1f0cf6b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:21.830163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbd2e8-2540-4785-a41b-a1b1f0cf6b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:21.830877 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbd2e8-2540-4785-a41b-a1b1f0cf6b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:21.832470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbd2e8-2540-4785-a41b-a1b1f0cf6b68 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:21.856418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 981/1959] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:41:21.860704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e2a36-36fd-4d93-8510-adbfd7a9b4ed admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:21.872877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e2a36-36fd-4d93-8510-adbfd7a9b4ed admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:21.880638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815771d-7b4b-40aa-b216-bfba5fb78bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:21.881503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815771d-7b4b-40aa-b216-bfba5fb78bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:21.881861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815771d-7b4b-40aa-b216-bfba5fb78bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:21.882247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815771d-7b4b-40aa-b216-bfba5fb78bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:21.883119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815771d-7b4b-40aa-b216-bfba5fb78bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:21.885818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815771d-7b4b-40aa-b216-bfba5fb78bb0 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:21.897841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 979/1960] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:21.903052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbd2e8-2540-4785-a41b-a1b1f0cf6b68 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:21.914362 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbd2e8-2540-4785-a41b-a1b1f0cf6b68 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:21.920280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e967d6-94d0-493a-a945-ab39eb29f277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:21.921015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e967d6-94d0-493a-a945-ab39eb29f277 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:21.921302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e967d6-94d0-493a-a945-ab39eb29f277 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:21.921600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e967d6-94d0-493a-a945-ab39eb29f277 admin admin] PATH_INFO: `/v3/projects/3f33d840d48242c897308d34b73ce05a/users/153b5eb059f74e83826b79c1cc8b83f2/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:21.939414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e967d6-94d0-493a-a945-ab39eb29f277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f33d840d48242c897308d34b73ce05a, user_id=153b5eb059f74e83826b79c1cc8b83f2, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:21.942127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e967d6-94d0-493a-a945-ab39eb29f277 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:22.041829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 982/1961] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:21 2022] PUT /identity/v3/projects/3f33d840d48242c897308d34b73ce05a/users/153b5eb059f74e83826b79c1cc8b83f2/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:41:22.053898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f5624-6e68-46d9-9d5a-f4a9aa2e0135 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:22.054216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f5624-6e68-46d9-9d5a-f4a9aa2e0135 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:22.054532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f5624-6e68-46d9-9d5a-f4a9aa2e0135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:22.084361 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ae1f5624-6e68-46d9-9d5a-f4a9aa2e0135 None None] MFA Rules not processed for user `153b5eb059f74e83826b79c1cc8b83f2`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:22.087273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ae1f5624-6e68-46d9-9d5a-f4a9aa2e0135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:22.154231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 980/1962] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:22.165735 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e967d6-94d0-493a-a945-ab39eb29f277 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:22.169429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e6e967d6-94d0-493a-a945-ab39eb29f277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:22.228495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e967d6-94d0-493a-a945-ab39eb29f277 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:22.234391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f006fe-a9fd-49d6-be59-71bc96ccf3f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:22.234391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f006fe-a9fd-49d6-be59-71bc96ccf3f7 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:22.234391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f006fe-a9fd-49d6-be59-71bc96ccf3f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:22.234391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f006fe-a9fd-49d6-be59-71bc96ccf3f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:22.244279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f006fe-a9fd-49d6-be59-71bc96ccf3f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:22.244279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f006fe-a9fd-49d6-be59-71bc96ccf3f7 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:22.253087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 983/1963] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:22.273187 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f5624-6e68-46d9-9d5a-f4a9aa2e0135 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:22.335785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f5624-6e68-46d9-9d5a-f4a9aa2e0135 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:22.341087 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:22.345463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:22.345463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:22.345463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] PATH_INFO: `/v3/users/153b5eb059f74e83826b79c1cc8b83f2` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:22.345463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] RBAC: Authorizing `identity:delete_user(user_id=153b5eb059f74e83826b79c1cc8b83f2)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:22.345463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:22.368136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:22.615034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '153b5eb059f74e83826b79c1cc8b83f2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:22.710138 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '153b5eb059f74e83826b79c1cc8b83f2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:22.710809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '153b5eb059f74e83826b79c1cc8b83f2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:22.726390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '153b5eb059f74e83826b79c1cc8b83f2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:22.735107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '153b5eb059f74e83826b79c1cc8b83f2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:22.744532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 981/1964] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:22 2022] DELETE /identity/v3/users/153b5eb059f74e83826b79c1cc8b83f2 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:22.749772 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdf921-fbce-4a5f-b997-0b9733fcbbed None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:22.753262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdf921-fbce-4a5f-b997-0b9733fcbbed None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:22.753995 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdf921-fbce-4a5f-b997-0b9733fcbbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:22.788565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-65bdf921-fbce-4a5f-b997-0b9733fcbbed None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:22.790109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-65bdf921-fbce-4a5f-b997-0b9733fcbbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:22.919012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 984/1965] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:22.961481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:22.964129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:23.109077 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f2248-cc03-4ba8-93c6-bfd4dd1ff72e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:23.114661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7f2cb-77f4-4041-bbfd-a373aa91e30e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:23.116165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7f2cb-77f4-4041-bbfd-a373aa91e30e service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:23.116702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7f2cb-77f4-4041-bbfd-a373aa91e30e service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:23.117446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7f2cb-77f4-4041-bbfd-a373aa91e30e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:23.130831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7f2cb-77f4-4041-bbfd-a373aa91e30e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:23.132298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7f2cb-77f4-4041-bbfd-a373aa91e30e service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:23.145044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 982/1966] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:23.659726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdf921-fbce-4a5f-b997-0b9733fcbbed None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:23.669956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-65bdf921-fbce-4a5f-b997-0b9733fcbbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:23.673364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdf921-fbce-4a5f-b997-0b9733fcbbed None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:23.679030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:23.679880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:23.680328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:23.680603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] PATH_INFO: `/v3/projects/3f33d840d48242c897308d34b73ce05a` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:23.689042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f33d840d48242c897308d34b73ce05a)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:23.691109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:23.816462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f33d840d48242c897308d34b73ce05a', 'request_id': 'req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:23.901534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f33d840d48242c897308d34b73ce05a', 'request_id': 'req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:23.906377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:23.908707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f33d840d48242c897308d34b73ce05a', 'request_id': 'req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:23.920062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] The token cache is being invalidate because project 3f33d840d48242c897308d34b73ce05a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:23.920715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:23.921052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ddb7187a-2cd3-4302-baba-2e6f58b7baa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:23.922928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 985/1967] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:23 2022] DELETE /identity/v3/projects/3f33d840d48242c897308d34b73ce05a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:23.927611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59889695-81bf-475e-bd66-95bc567d4fd8 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:23.928173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59889695-81bf-475e-bd66-95bc567d4fd8 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:23.928173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59889695-81bf-475e-bd66-95bc567d4fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:23.953035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-59889695-81bf-475e-bd66-95bc567d4fd8 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:23.954301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-59889695-81bf-475e-bd66-95bc567d4fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:24.031347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 983/1968] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:41:24.039957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58049e-86fc-4b79-9012-bbe80115f30f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:24.043898 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58049e-86fc-4b79-9012-bbe80115f30f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:24.043898 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58049e-86fc-4b79-9012-bbe80115f30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:24.074853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ef58049e-86fc-4b79-9012-bbe80115f30f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:24.076229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ef58049e-86fc-4b79-9012-bbe80115f30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:24.087179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 986/1969] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:41:24.089767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59889695-81bf-475e-bd66-95bc567d4fd8 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:24.100514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59889695-81bf-475e-bd66-95bc567d4fd8 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:24.107779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4928465-6ad2-4ca2-a24c-3ac17ae244d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:24.108483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4928465-6ad2-4ca2-a24c-3ac17ae244d1 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:24.109408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4928465-6ad2-4ca2-a24c-3ac17ae244d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:24.109513 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4928465-6ad2-4ca2-a24c-3ac17ae244d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:24.110350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4928465-6ad2-4ca2-a24c-3ac17ae244d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:41:24.112999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4928465-6ad2-4ca2-a24c-3ac17ae244d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:24.112999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4928465-6ad2-4ca2-a24c-3ac17ae244d1 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:24.130182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 984/1970] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:41:24.133715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58049e-86fc-4b79-9012-bbe80115f30f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:24.145140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58049e-86fc-4b79-9012-bbe80115f30f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:24.151720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb2571-fbf4-42ec-8f27-c8456b8eea04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:24.152688 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb2571-fbf4-42ec-8f27-c8456b8eea04 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:24.153409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb2571-fbf4-42ec-8f27-c8456b8eea04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:24.153912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb2571-fbf4-42ec-8f27-c8456b8eea04 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:24.154805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb2571-fbf4-42ec-8f27-c8456b8eea04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:24.157048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb2571-fbf4-42ec-8f27-c8456b8eea04 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:24.334450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 987/1971] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:41:24 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:24.339590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4928465-6ad2-4ca2-a24c-3ac17ae244d1 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:24.350673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4928465-6ad2-4ca2-a24c-3ac17ae244d1 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:24.392851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:24.423606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e840fce-8389-4c85-a4a6-6891cf54bce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:24.424284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e840fce-8389-4c85-a4a6-6891cf54bce4 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:24.424569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e840fce-8389-4c85-a4a6-6891cf54bce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:24.424842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e840fce-8389-4c85-a4a6-6891cf54bce4 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:24.425521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e840fce-8389-4c85-a4a6-6891cf54bce4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:24.427532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e840fce-8389-4c85-a4a6-6891cf54bce4 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:24.547314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 985/1972] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:41:24 2022] POST /identity/v3/users => generated 287 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:24.556595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb2571-fbf4-42ec-8f27-c8456b8eea04 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:24.564342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb2571-fbf4-42ec-8f27-c8456b8eea04 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:24.569115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359782c-fd9f-43b2-9194-c43c2a1172c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:24.569732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359782c-fd9f-43b2-9194-c43c2a1172c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:24.570033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359782c-fd9f-43b2-9194-c43c2a1172c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:24.570330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359782c-fd9f-43b2-9194-c43c2a1172c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:24.571021 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359782c-fd9f-43b2-9194-c43c2a1172c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:24.573221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359782c-fd9f-43b2-9194-c43c2a1172c1 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:24.584778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 988/1973] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:41:24.588928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e840fce-8389-4c85-a4a6-6891cf54bce4 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:24.599685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e840fce-8389-4c85-a4a6-6891cf54bce4 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:24.604530 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637936f1-fe1c-4756-baf4-ea25bb9fbd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:24.605205 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637936f1-fe1c-4756-baf4-ea25bb9fbd70 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:24.605496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637936f1-fe1c-4756-baf4-ea25bb9fbd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:24.605773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637936f1-fe1c-4756-baf4-ea25bb9fbd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:24.606511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637936f1-fe1c-4756-baf4-ea25bb9fbd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:24.608109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637936f1-fe1c-4756-baf4-ea25bb9fbd70 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:24.620386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 986/1974] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:24.624902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359782c-fd9f-43b2-9194-c43c2a1172c1 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:24.635689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359782c-fd9f-43b2-9194-c43c2a1172c1 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:24.641296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819c2bd-16ce-4b77-8e72-508bff0d5b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:24.641970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819c2bd-16ce-4b77-8e72-508bff0d5b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:24.642287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819c2bd-16ce-4b77-8e72-508bff0d5b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:24.642565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819c2bd-16ce-4b77-8e72-508bff0d5b42 admin admin] PATH_INFO: `/v3/projects/3fac2bc3c2704357b1e76099e718a843/users/52d4836095e744cbae8a4989801a58e2/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:24.658740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819c2bd-16ce-4b77-8e72-508bff0d5b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fac2bc3c2704357b1e76099e718a843, user_id=52d4836095e744cbae8a4989801a58e2, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:24.661420 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819c2bd-16ce-4b77-8e72-508bff0d5b42 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:24.745430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 989/1975] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:24 2022] PUT /identity/v3/projects/3fac2bc3c2704357b1e76099e718a843/users/52d4836095e744cbae8a4989801a58e2/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:24.750619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbf9df-653f-490d-acdf-42d4606d2311 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:24.751106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbf9df-653f-490d-acdf-42d4606d2311 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:24.751551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbf9df-653f-490d-acdf-42d4606d2311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:24.797723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-9ccbf9df-653f-490d-acdf-42d4606d2311 None None] MFA Rules not processed for user `52d4836095e744cbae8a4989801a58e2`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:24.798992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9ccbf9df-653f-490d-acdf-42d4606d2311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:24.872820 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 987/1976] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:24.883924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819c2bd-16ce-4b77-8e72-508bff0d5b42 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:24.886490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5819c2bd-16ce-4b77-8e72-508bff0d5b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:24.946584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819c2bd-16ce-4b77-8e72-508bff0d5b42 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:24.950499 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8635c-206a-4167-b3b9-403da52306e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:24.951121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8635c-206a-4167-b3b9-403da52306e3 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:24.951510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8635c-206a-4167-b3b9-403da52306e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:24.951838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8635c-206a-4167-b3b9-403da52306e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:24.960342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8635c-206a-4167-b3b9-403da52306e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:24.961743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8635c-206a-4167-b3b9-403da52306e3 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:24.971117 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 990/1977] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:34.662110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbf9df-653f-490d-acdf-42d4606d2311 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:34.726982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbf9df-653f-490d-acdf-42d4606d2311 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:34.735728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:34.736512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:34.736891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:34.737168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] PATH_INFO: `/v3/users/52d4836095e744cbae8a4989801a58e2` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:34.738726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d4836095e744cbae8a4989801a58e2)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:34.740939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:34.931515 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:34.937164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d4836095e744cbae8a4989801a58e2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:34.966619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=1,keystone:SELECT=57,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:35.033030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d4836095e744cbae8a4989801a58e2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:35.034068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d4836095e744cbae8a4989801a58e2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:35.071368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d4836095e744cbae8a4989801a58e2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:35.095921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d4836095e744cbae8a4989801a58e2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:35.105163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 988/1978] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:34 2022] DELETE /identity/v3/users/52d4836095e744cbae8a4989801a58e2 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:35.130145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b933a-e8d7-49fc-9333-52461ff5f759 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:35.133856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b933a-e8d7-49fc-9333-52461ff5f759 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:35.133856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b933a-e8d7-49fc-9333-52461ff5f759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:35.161460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-119b933a-e8d7-49fc-9333-52461ff5f759 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:35.163132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-119b933a-e8d7-49fc-9333-52461ff5f759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:35.242850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 991/1979] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:35.254869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:35.256977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:35.338815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0696be-c6d2-4831-81d9-55f7ca232c4c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:35.342807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d3596-7882-4284-a636-d3b394717794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:35.343432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d3596-7882-4284-a636-d3b394717794 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:35.343756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d3596-7882-4284-a636-d3b394717794 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:35.344033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d3596-7882-4284-a636-d3b394717794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:35.353266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d3596-7882-4284-a636-d3b394717794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:35.355331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d3596-7882-4284-a636-d3b394717794 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:35.365080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 989/1980] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:35.804804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b933a-e8d7-49fc-9333-52461ff5f759 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:35.806606 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-119b933a-e8d7-49fc-9333-52461ff5f759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:35.820226 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b933a-e8d7-49fc-9333-52461ff5f759 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:35.826382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:35.827414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:35.827801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:35.828184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] PATH_INFO: `/v3/projects/3fac2bc3c2704357b1e76099e718a843` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:35.830164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fac2bc3c2704357b1e76099e718a843)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:35.834265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:35.968289 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fac2bc3c2704357b1e76099e718a843', 'request_id': 'req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:36.044455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fac2bc3c2704357b1e76099e718a843', 'request_id': 'req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:36.050970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fac2bc3c2704357b1e76099e718a843', 'request_id': 'req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:36.063291 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] The token cache is being invalidate because project 3fac2bc3c2704357b1e76099e718a843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:36.063578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:36.064231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-d9b3bc18-6b16-43e5-86c3-a9d45931e13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:36.066226 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 992/1981] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:35 2022] DELETE /identity/v3/projects/3fac2bc3c2704357b1e76099e718a843 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:36.070873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373b169-1de6-4e12-9830-6d4d7b27eeec None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:36.071361 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373b169-1de6-4e12-9830-6d4d7b27eeec None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:36.071447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373b169-1de6-4e12-9830-6d4d7b27eeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:36.093120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1373b169-1de6-4e12-9830-6d4d7b27eeec None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:36.094469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1373b169-1de6-4e12-9830-6d4d7b27eeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:36.163334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 990/1982] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:36.172544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d4f27-784f-4da9-aa37-7f499c2d2d7d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:36.172831 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d4f27-784f-4da9-aa37-7f499c2d2d7d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:36.173128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d4f27-784f-4da9-aa37-7f499c2d2d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:36.198426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-3d9d4f27-784f-4da9-aa37-7f499c2d2d7d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:36.201589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3d9d4f27-784f-4da9-aa37-7f499c2d2d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:36.210662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 993/1983] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:36.215510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373b169-1de6-4e12-9830-6d4d7b27eeec None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:36.236338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373b169-1de6-4e12-9830-6d4d7b27eeec None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:36.248710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9040dc-5794-494d-8656-b8df6f38bf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:36.250713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9040dc-5794-494d-8656-b8df6f38bf7b admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:36.251008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9040dc-5794-494d-8656-b8df6f38bf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:36.251321 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9040dc-5794-494d-8656-b8df6f38bf7b admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:36.252016 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9040dc-5794-494d-8656-b8df6f38bf7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:41:36.252328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9040dc-5794-494d-8656-b8df6f38bf7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:36.255223 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9040dc-5794-494d-8656-b8df6f38bf7b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:36.295372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 991/1984] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:41:36.296414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d4f27-784f-4da9-aa37-7f499c2d2d7d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:36.311046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d4f27-784f-4da9-aa37-7f499c2d2d7d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:36.328047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21de53-1412-4011-8b05-9c03177f40d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:36.328680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21de53-1412-4011-8b05-9c03177f40d2 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:36.328964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21de53-1412-4011-8b05-9c03177f40d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:36.329426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21de53-1412-4011-8b05-9c03177f40d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:36.330637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21de53-1412-4011-8b05-9c03177f40d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:36.332546 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21de53-1412-4011-8b05-9c03177f40d2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:36.459140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 994/1985] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:41:36 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:36.470363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9040dc-5794-494d-8656-b8df6f38bf7b admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:36.490873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9040dc-5794-494d-8656-b8df6f38bf7b admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:36.585833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21de53-1412-4011-8b05-9c03177f40d2 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:36.587773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-0e21de53-1412-4011-8b05-9c03177f40d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:36.594303 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:36.612777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f403d-bc78-4187-99fb-3793f5e61bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:36.614404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f403d-bc78-4187-99fb-3793f5e61bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:36.615119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f403d-bc78-4187-99fb-3793f5e61bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:36.615713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f403d-bc78-4187-99fb-3793f5e61bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:36.616668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f403d-bc78-4187-99fb-3793f5e61bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:36.620923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f403d-bc78-4187-99fb-3793f5e61bc1 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:36.691465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:36.695260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21de53-1412-4011-8b05-9c03177f40d2 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:36.700905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e415447-425f-426e-98a0-83e3ecc12cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:36.701565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e415447-425f-426e-98a0-83e3ecc12cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:36.701849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e415447-425f-426e-98a0-83e3ecc12cec admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:36.702164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e415447-425f-426e-98a0-83e3ecc12cec admin admin] PATH_INFO: `/v3/users/851ecae88d3c440db2fc68ab16a700ee` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:36.703354 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e415447-425f-426e-98a0-83e3ecc12cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=851ecae88d3c440db2fc68ab16a700ee)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:36.705331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e415447-425f-426e-98a0-83e3ecc12cec admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:36.741922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 992/1986] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:41:36 2022] POST /identity/v3/users => generated 292 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:36.746337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f403d-bc78-4187-99fb-3793f5e61bc1 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:36.756791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f403d-bc78-4187-99fb-3793f5e61bc1 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:36.762065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d8f08-5d3d-457f-936b-5c0cabe8500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:36.762729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8f08-5d3d-457f-936b-5c0cabe8500d admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:36.763013 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8f08-5d3d-457f-936b-5c0cabe8500d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:36.763292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8f08-5d3d-457f-936b-5c0cabe8500d admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:36.764134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d8f08-5d3d-457f-936b-5c0cabe8500d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:36.765786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d8f08-5d3d-457f-936b-5c0cabe8500d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:36.781494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 993/1987] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:41:36.785345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d8f08-5d3d-457f-936b-5c0cabe8500d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:36.796234 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d8f08-5d3d-457f-936b-5c0cabe8500d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:36.801085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad2e70-7b42-4df4-9ae3-71678c420e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:36.801624 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad2e70-7b42-4df4-9ae3-71678c420e76 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:36.801910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad2e70-7b42-4df4-9ae3-71678c420e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:36.802228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad2e70-7b42-4df4-9ae3-71678c420e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:36.802923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad2e70-7b42-4df4-9ae3-71678c420e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:36.804546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad2e70-7b42-4df4-9ae3-71678c420e76 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:36.827495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 994/1988] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:41:36.829592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad2e70-7b42-4df4-9ae3-71678c420e76 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:36.844314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad2e70-7b42-4df4-9ae3-71678c420e76 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:36.849219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c0d80-354e-4014-bcff-4d008145c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:36.849887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c0d80-354e-4014-bcff-4d008145c20f admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:36.850200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c0d80-354e-4014-bcff-4d008145c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:36.850479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c0d80-354e-4014-bcff-4d008145c20f admin admin] PATH_INFO: `/v3/projects/38df81c808584094bdbd66b0b88582b9/users/ded9c5fc23184944a8a782e3f9755154/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:36.865620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c0d80-354e-4014-bcff-4d008145c20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=38df81c808584094bdbd66b0b88582b9, user_id=ded9c5fc23184944a8a782e3f9755154, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:36.868576 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c0d80-354e-4014-bcff-4d008145c20f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:36.910099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8e415447-425f-426e-98a0-83e3ecc12cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '851ecae88d3c440db2fc68ab16a700ee'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:36.953825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 995/1989] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:36 2022] PUT /identity/v3/projects/38df81c808584094bdbd66b0b88582b9/users/ded9c5fc23184944a8a782e3f9755154/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:36.967843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc321a-efa0-4127-8591-0864cf3af5f5 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:36.968311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc321a-efa0-4127-8591-0864cf3af5f5 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:36.968697 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc321a-efa0-4127-8591-0864cf3af5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:36.997810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-41fc321a-efa0-4127-8591-0864cf3af5f5 None None] MFA Rules not processed for user `ded9c5fc23184944a8a782e3f9755154`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:36.999080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-41fc321a-efa0-4127-8591-0864cf3af5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:37.003832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8e415447-425f-426e-98a0-83e3ecc12cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '851ecae88d3c440db2fc68ab16a700ee'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:37.019073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8e415447-425f-426e-98a0-83e3ecc12cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '851ecae88d3c440db2fc68ab16a700ee'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:37.019837 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8e415447-425f-426e-98a0-83e3ecc12cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '851ecae88d3c440db2fc68ab16a700ee'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:37.027126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8e415447-425f-426e-98a0-83e3ecc12cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '851ecae88d3c440db2fc68ab16a700ee'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:37.040422 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 995/1990] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:36 2022] DELETE /identity/v3/users/851ecae88d3c440db2fc68ab16a700ee => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:41:37.045359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85606418-e25d-4777-82bd-c7f01bf8f592 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:37.045359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85606418-e25d-4777-82bd-c7f01bf8f592 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:37.045838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85606418-e25d-4777-82bd-c7f01bf8f592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:37.088111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-85606418-e25d-4777-82bd-c7f01bf8f592 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:37.089345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-85606418-e25d-4777-82bd-c7f01bf8f592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:37.093734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 996/1991] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:41:37.106073 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc321a-efa0-4127-8591-0864cf3af5f5 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:37.108556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-41fc321a-efa0-4127-8591-0864cf3af5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:37.156828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 996/1992] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:37.167313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85606418-e25d-4777-82bd-c7f01bf8f592 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:37.168656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-85606418-e25d-4777-82bd-c7f01bf8f592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:37.177240 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc321a-efa0-4127-8591-0864cf3af5f5 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:37.211052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:37.235873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85606418-e25d-4777-82bd-c7f01bf8f592 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:37.237076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45323ca9-20ea-4d52-89b1-7ee4c602cbd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:37.237708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323ca9-20ea-4d52-89b1-7ee4c602cbd1 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:37.238019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323ca9-20ea-4d52-89b1-7ee4c602cbd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:37.240672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323ca9-20ea-4d52-89b1-7ee4c602cbd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:37.240778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66e089-3fd7-44cd-ae97-71dec2e29fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:37.240778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66e089-3fd7-44cd-ae97-71dec2e29fec service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:37.240778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66e089-3fd7-44cd-ae97-71dec2e29fec service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:37.241033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66e089-3fd7-44cd-ae97-71dec2e29fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:37.251152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66e089-3fd7-44cd-ae97-71dec2e29fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:37.252589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66e089-3fd7-44cd-ae97-71dec2e29fec service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:37.263015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 997/1993] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:41:37.299132 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45323ca9-20ea-4d52-89b1-7ee4c602cbd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:37.300953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45323ca9-20ea-4d52-89b1-7ee4c602cbd1 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:37.330409 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 997/1994] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:41:37.897214 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66e089-3fd7-44cd-ae97-71dec2e29fec service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:37.898680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6e66e089-3fd7-44cd-ae97-71dec2e29fec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:37.919180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66e089-3fd7-44cd-ae97-71dec2e29fec service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:37.924900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:37.925562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:37.925842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:37.926139 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] PATH_INFO: `/v3/projects/b7f2a60a8bd04ec092e5794dbbf48b84` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:37.927439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f2a60a8bd04ec092e5794dbbf48b84)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:37.929464 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:37.943316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:38.065017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f2a60a8bd04ec092e5794dbbf48b84', 'request_id': 'req-67da810e-67c7-4824-a189-5f1443fa078c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:38.143742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f2a60a8bd04ec092e5794dbbf48b84', 'request_id': 'req-67da810e-67c7-4824-a189-5f1443fa078c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:38.152112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f2a60a8bd04ec092e5794dbbf48b84', 'request_id': 'req-67da810e-67c7-4824-a189-5f1443fa078c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:38.163944 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] The token cache is being invalidate because project b7f2a60a8bd04ec092e5794dbbf48b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:38.164550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:38.165148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-67da810e-67c7-4824-a189-5f1443fa078c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:38.171221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 998/1995] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:37 2022] DELETE /identity/v3/projects/b7f2a60a8bd04ec092e5794dbbf48b84 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:38.176322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e291e-591e-446f-a4bd-acbfdce186ca None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:38.176322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e291e-591e-446f-a4bd-acbfdce186ca None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:38.176322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e291e-591e-446f-a4bd-acbfdce186ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:38.196321 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c61e291e-591e-446f-a4bd-acbfdce186ca None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:38.198759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c61e291e-591e-446f-a4bd-acbfdce186ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:38.262735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 998/1996] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:38.270716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb82a44-8a74-4a61-bca3-d8d95b14b2b4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:38.271009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb82a44-8a74-4a61-bca3-d8d95b14b2b4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:38.271288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb82a44-8a74-4a61-bca3-d8d95b14b2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:38.292521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-3bb82a44-8a74-4a61-bca3-d8d95b14b2b4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:38.293743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3bb82a44-8a74-4a61-bca3-d8d95b14b2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:38.302155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 999/1997] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:38.306389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e291e-591e-446f-a4bd-acbfdce186ca None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:38.317393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e291e-591e-446f-a4bd-acbfdce186ca None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:38.322356 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738061e9-2a10-4193-ae6f-03454c12099f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:38.323337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738061e9-2a10-4193-ae6f-03454c12099f admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:38.323628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738061e9-2a10-4193-ae6f-03454c12099f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:38.324922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738061e9-2a10-4193-ae6f-03454c12099f admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:38.324922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738061e9-2a10-4193-ae6f-03454c12099f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:41:38.324922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738061e9-2a10-4193-ae6f-03454c12099f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:38.326626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738061e9-2a10-4193-ae6f-03454c12099f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:38.338814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 999/1998] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:41:38.343169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb82a44-8a74-4a61-bca3-d8d95b14b2b4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:38.354778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb82a44-8a74-4a61-bca3-d8d95b14b2b4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:38.360811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607c229-7d0a-4635-8cce-cf71b4120ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:38.361530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607c229-7d0a-4635-8cce-cf71b4120ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:38.361823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607c229-7d0a-4635-8cce-cf71b4120ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:38.362126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607c229-7d0a-4635-8cce-cf71b4120ef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:38.362802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607c229-7d0a-4635-8cce-cf71b4120ef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:38.364684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607c229-7d0a-4635-8cce-cf71b4120ef6 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:38.493628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1000/1999] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:41:38 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:38.497711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738061e9-2a10-4193-ae6f-03454c12099f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:38.508115 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738061e9-2a10-4193-ae6f-03454c12099f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:38.569307 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6167f1a-875d-4eb9-8345-ed6503ba0af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:38.570130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6167f1a-875d-4eb9-8345-ed6503ba0af5 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:38.570413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6167f1a-875d-4eb9-8345-ed6503ba0af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:38.570687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6167f1a-875d-4eb9-8345-ed6503ba0af5 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:38.571362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6167f1a-875d-4eb9-8345-ed6503ba0af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:38.574042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6167f1a-875d-4eb9-8345-ed6503ba0af5 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:38.641058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1000/2000] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:41:38 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:41:38.641264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607c229-7d0a-4635-8cce-cf71b4120ef6 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:38.650774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607c229-7d0a-4635-8cce-cf71b4120ef6 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:38.656900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7090d4-cbe6-4b53-a582-e57d14678737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:38.657640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7090d4-cbe6-4b53-a582-e57d14678737 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:38.662736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7090d4-cbe6-4b53-a582-e57d14678737 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:38.662736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7090d4-cbe6-4b53-a582-e57d14678737 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:38.662736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7090d4-cbe6-4b53-a582-e57d14678737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:38.662736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7090d4-cbe6-4b53-a582-e57d14678737 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:38.675226 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1001/2001] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:38.676580 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6167f1a-875d-4eb9-8345-ed6503ba0af5 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:38.687745 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6167f1a-875d-4eb9-8345-ed6503ba0af5 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:38.692373 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3bb79-59a7-475d-ac30-c85b3a13b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:38.693209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3bb79-59a7-475d-ac30-c85b3a13b6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:38.693303 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3bb79-59a7-475d-ac30-c85b3a13b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:38.693505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3bb79-59a7-475d-ac30-c85b3a13b6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:38.694225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3bb79-59a7-475d-ac30-c85b3a13b6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:38.695806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3bb79-59a7-475d-ac30-c85b3a13b6a9 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:38.698819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:38.708212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1001/2002] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:41:38.712388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7090d4-cbe6-4b53-a582-e57d14678737 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:38.722894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7090d4-cbe6-4b53-a582-e57d14678737 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:38.727779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a5113-18ec-4285-8b60-f0edd19a0744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:38.728811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a5113-18ec-4285-8b60-f0edd19a0744 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:38.729220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a5113-18ec-4285-8b60-f0edd19a0744 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:38.729585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a5113-18ec-4285-8b60-f0edd19a0744 admin admin] PATH_INFO: `/v3/projects/b37a8c7b37ac4f8d9711ac8cd0e37f02/users/db4a45e9f40740c99cc49d9f2978332d/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:38.747379 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a5113-18ec-4285-8b60-f0edd19a0744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b37a8c7b37ac4f8d9711ac8cd0e37f02, user_id=db4a45e9f40740c99cc49d9f2978332d, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:38.750234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a5113-18ec-4285-8b60-f0edd19a0744 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:38.841156 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1002/2003] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:41:38 2022] PUT /identity/v3/projects/b37a8c7b37ac4f8d9711ac8cd0e37f02/users/db4a45e9f40740c99cc49d9f2978332d/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:41:38.856730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb73762-9fb7-4469-8962-fb5cf3a0ac23 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:38.857016 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb73762-9fb7-4469-8962-fb5cf3a0ac23 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:38.857289 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb73762-9fb7-4469-8962-fb5cf3a0ac23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:38.880629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-9cb73762-9fb7-4469-8962-fb5cf3a0ac23 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:38.881869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9cb73762-9fb7-4469-8962-fb5cf3a0ac23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:38.960408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1002/2004] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:38.975648 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a5113-18ec-4285-8b60-f0edd19a0744 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:38.977484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-9f7a5113-18ec-4285-8b60-f0edd19a0744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:39.057356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a5113-18ec-4285-8b60-f0edd19a0744 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:39.061948 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3387086-3cdc-4c7b-a819-c02463b6883d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:39.062895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3387086-3cdc-4c7b-a819-c02463b6883d service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:39.063533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3387086-3cdc-4c7b-a819-c02463b6883d service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:39.063533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3387086-3cdc-4c7b-a819-c02463b6883d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:39.073760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3387086-3cdc-4c7b-a819-c02463b6883d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:39.075226 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3387086-3cdc-4c7b-a819-c02463b6883d service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:39.085150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1003/2005] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:40.320883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230139c-08ff-4bce-b967-337d7c039f25 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:40.321400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230139c-08ff-4bce-b967-337d7c039f25 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:40.321572 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230139c-08ff-4bce-b967-337d7c039f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:40.351320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-8230139c-08ff-4bce-b967-337d7c039f25 None None] MFA Rules not processed for user `db4a45e9f40740c99cc49d9f2978332d`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:40.363476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8230139c-08ff-4bce-b967-337d7c039f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:40.425318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1003/2006] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:40.436997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3387086-3cdc-4c7b-a819-c02463b6883d service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:40.438482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e3387086-3cdc-4c7b-a819-c02463b6883d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:40.483094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:40.496233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3387086-3cdc-4c7b-a819-c02463b6883d service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:40.499816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28caf870-e1a0-446e-aa9d-9e32f12279b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:40.500438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caf870-e1a0-446e-aa9d-9e32f12279b5 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:40.500712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caf870-e1a0-446e-aa9d-9e32f12279b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:40.500994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caf870-e1a0-446e-aa9d-9e32f12279b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:40.509046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28caf870-e1a0-446e-aa9d-9e32f12279b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:40.511011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28caf870-e1a0-446e-aa9d-9e32f12279b5 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:40.520138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1004/2007] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:50.421798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=60 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:50.576983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=8 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:41:50.693752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173f8c0-72c5-4cf8-bf4c-c17d8ee0a562 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:50.694182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173f8c0-72c5-4cf8-bf4c-c17d8ee0a562 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:50.694606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173f8c0-72c5-4cf8-bf4c-c17d8ee0a562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:50.715783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1173f8c0-72c5-4cf8-bf4c-c17d8ee0a562 None None] MFA Rules not processed for user `db4a45e9f40740c99cc49d9f2978332d`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:50.717012 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1173f8c0-72c5-4cf8-bf4c-c17d8ee0a562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:50.730473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1004/2008] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:52.085144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28caf870-e1a0-446e-aa9d-9e32f12279b5 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:52.087353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-28caf870-e1a0-446e-aa9d-9e32f12279b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:52.112271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28caf870-e1a0-446e-aa9d-9e32f12279b5 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:52.122964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:52.122964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:52.122964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:52.122964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] PATH_INFO: `/v3/users/ded9c5fc23184944a8a782e3f9755154` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:52.125146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded9c5fc23184944a8a782e3f9755154)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:52.129252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:52.343728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded9c5fc23184944a8a782e3f9755154'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:52.418725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded9c5fc23184944a8a782e3f9755154'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:52.461056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded9c5fc23184944a8a782e3f9755154'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:52.462555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded9c5fc23184944a8a782e3f9755154'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:52.472160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded9c5fc23184944a8a782e3f9755154'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:52.479035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1005/2009] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:41:52 2022] DELETE /identity/v3/users/ded9c5fc23184944a8a782e3f9755154 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:41:52.484420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caba26-fe21-4bae-9e18-b7819c3a62ca None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:52.484924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caba26-fe21-4bae-9e18-b7819c3a62ca None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:52.485075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caba26-fe21-4bae-9e18-b7819c3a62ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:52.506781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a8caba26-fe21-4bae-9e18-b7819c3a62ca None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:41:52.508288 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a8caba26-fe21-4bae-9e18-b7819c3a62ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:52.584518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1005/2010] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:41:52.600953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:52.603550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:52.727612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd20d5f-5bc2-4267-aae4-d522ee2c3433 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:52.733149 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987aeec-e391-46f4-952f-0e80993f874c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:52.734107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987aeec-e391-46f4-952f-0e80993f874c service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:52.734553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987aeec-e391-46f4-952f-0e80993f874c service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:52.735000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987aeec-e391-46f4-952f-0e80993f874c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:52.749318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987aeec-e391-46f4-952f-0e80993f874c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:52.751776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987aeec-e391-46f4-952f-0e80993f874c service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:52.770064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1006/2011] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:41:53.755824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caba26-fe21-4bae-9e18-b7819c3a62ca None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:41:53.758518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a8caba26-fe21-4bae-9e18-b7819c3a62ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:41:53.787090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caba26-fe21-4bae-9e18-b7819c3a62ca None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:41:53.794480 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:41:53.795913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:41:53.796417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:41:53.796940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] PATH_INFO: `/v3/projects/38df81c808584094bdbd66b0b88582b9` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:41:53.798615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=38df81c808584094bdbd66b0b88582b9)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:41:53.801054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:41:54.045528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38df81c808584094bdbd66b0b88582b9', 'request_id': 'req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:54.068818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38df81c808584094bdbd66b0b88582b9', 'request_id': 'req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:54.211928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38df81c808584094bdbd66b0b88582b9', 'request_id': 'req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:54.224647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] The token cache is being invalidate because project 38df81c808584094bdbd66b0b88582b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:41:54.225055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:54.226094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-85bbdb3a-4059-4f91-ae9e-640cd35d4c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:41:54.227912 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1006/2012] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:41:53 2022] DELETE /identity/v3/projects/38df81c808584094bdbd66b0b88582b9 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:42:02.758306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:04.220974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:13.265092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987aeec-e391-46f4-952f-0e80993f874c service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:13.266563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6987aeec-e391-46f4-952f-0e80993f874c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:13.281612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987aeec-e391-46f4-952f-0e80993f874c service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:13.340097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:13.341363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:13.341649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:13.341928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] PATH_INFO: `/v3/users/36ef8cd66e3443979b2f312cb680a362` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:13.343129 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=36ef8cd66e3443979b2f312cb680a362)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:13.345032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:13.589305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36ef8cd66e3443979b2f312cb680a362'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:13.832103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36ef8cd66e3443979b2f312cb680a362'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:13.845415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36ef8cd66e3443979b2f312cb680a362'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:13.845997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36ef8cd66e3443979b2f312cb680a362'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:13.852605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36ef8cd66e3443979b2f312cb680a362'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:13.858522 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1007/2013] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:13 2022] DELETE /identity/v3/users/36ef8cd66e3443979b2f312cb680a362 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:13.863089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0208e-3f61-4254-9f99-431cbf18ff23 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:13.863605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0208e-3f61-4254-9f99-431cbf18ff23 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:13.863709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0208e-3f61-4254-9f99-431cbf18ff23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:13.883060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-6af0208e-3f61-4254-9f99-431cbf18ff23 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:13.884361 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6af0208e-3f61-4254-9f99-431cbf18ff23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:13.942273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1007/2014] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:13.952908 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:13.954438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:14.018049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7550d-6226-4d08-ac48-d97c518698aa admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:14.021712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdfa1c-7788-41ed-958d-5d3c275764f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:14.022483 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdfa1c-7788-41ed-958d-5d3c275764f7 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:14.022840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdfa1c-7788-41ed-958d-5d3c275764f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:14.023183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdfa1c-7788-41ed-958d-5d3c275764f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:14.033914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdfa1c-7788-41ed-958d-5d3c275764f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:14.035760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdfa1c-7788-41ed-958d-5d3c275764f7 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:14.046286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1008/2015] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:42:14.560298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0208e-3f61-4254-9f99-431cbf18ff23 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:14.561825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6af0208e-3f61-4254-9f99-431cbf18ff23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:14.576798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0208e-3f61-4254-9f99-431cbf18ff23 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:14.581647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:14.582347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:14.582633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:14.582909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] PATH_INFO: `/v3/projects/b64126c6313746c09e07f06369c64711` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:14.584210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b64126c6313746c09e07f06369c64711)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:14.586103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:14.703915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b64126c6313746c09e07f06369c64711', 'request_id': 'req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:14.710085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b64126c6313746c09e07f06369c64711', 'request_id': 'req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:14.792255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b64126c6313746c09e07f06369c64711', 'request_id': 'req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:14.803695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] The token cache is being invalidate because project b64126c6313746c09e07f06369c64711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:14.804027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:14.804684 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:14.806249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1008/2016] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:14 2022] DELETE /identity/v3/projects/b64126c6313746c09e07f06369c64711 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:14.810451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdfa1c-7788-41ed-958d-5d3c275764f7 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:14.819987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdfa1c-7788-41ed-958d-5d3c275764f7 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:14.838073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:14.877902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f304fd-080e-47a6-b72b-fdf79645bad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:14.878599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f304fd-080e-47a6-b72b-fdf79645bad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:14.878892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f304fd-080e-47a6-b72b-fdf79645bad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:14.879170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f304fd-080e-47a6-b72b-fdf79645bad5 admin admin] PATH_INFO: `/v3/users/4f1d27ebd7474643bfa2543ef1af3bf1` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:14.880359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f304fd-080e-47a6-b72b-fdf79645bad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f1d27ebd7474643bfa2543ef1af3bf1)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:14.882378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f304fd-080e-47a6-b72b-fdf79645bad5 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:15.062521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-11f304fd-080e-47a6-b72b-fdf79645bad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f1d27ebd7474643bfa2543ef1af3bf1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:15.137090 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-11f304fd-080e-47a6-b72b-fdf79645bad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f1d27ebd7474643bfa2543ef1af3bf1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:15.152194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-11f304fd-080e-47a6-b72b-fdf79645bad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f1d27ebd7474643bfa2543ef1af3bf1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:15.157751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-11f304fd-080e-47a6-b72b-fdf79645bad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f1d27ebd7474643bfa2543ef1af3bf1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:15.160790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-11f304fd-080e-47a6-b72b-fdf79645bad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f1d27ebd7474643bfa2543ef1af3bf1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:15.167592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1009/2017] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:14 2022] DELETE /identity/v3/users/4f1d27ebd7474643bfa2543ef1af3bf1 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:15.675444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:15.676954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:15.692249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2a270-b36e-4e8e-aef0-c70732bb8d17 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:15.751018 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:15.751660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:15.751946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:15.752222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] PATH_INFO: `/v3/projects/fe49d009c94d431ea12286d8754fd383` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:15.753388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe49d009c94d431ea12286d8754fd383)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:15.755338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:15.846393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:15.862840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe49d009c94d431ea12286d8754fd383', 'request_id': 'req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:15.869176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe49d009c94d431ea12286d8754fd383', 'request_id': 'req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:15.919506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe49d009c94d431ea12286d8754fd383', 'request_id': 'req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:15.929914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] The token cache is being invalidate because project fe49d009c94d431ea12286d8754fd383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:15.930596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:15.932146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3cc31c23-84d2-49f4-9d0c-4ccfdbdbe4c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:15.933974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1009/2018] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:15 2022] DELETE /identity/v3/projects/fe49d009c94d431ea12286d8754fd383 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:15.939155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63bcc1-6f66-46c4-b97c-e7584c841112 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:15.939620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63bcc1-6f66-46c4-b97c-e7584c841112 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:15.939814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63bcc1-6f66-46c4-b97c-e7584c841112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:15.959531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-dc63bcc1-6f66-46c4-b97c-e7584c841112 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:15.964157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-dc63bcc1-6f66-46c4-b97c-e7584c841112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:16.019802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1010/2019] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:16.027390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1421c7-c8c2-4950-9856-12ee10fc7168 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:16.027675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1421c7-c8c2-4950-9856-12ee10fc7168 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:16.027946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1421c7-c8c2-4950-9856-12ee10fc7168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:16.048747 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ed1421c7-c8c2-4950-9856-12ee10fc7168 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:16.049981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ed1421c7-c8c2-4950-9856-12ee10fc7168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:16.057913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1010/2020] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:16.062083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63bcc1-6f66-46c4-b97c-e7584c841112 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:16.071611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63bcc1-6f66-46c4-b97c-e7584c841112 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:16.076905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f684c8-d0d2-4dc5-baa7-a9c3d87dff61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:16.077474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f684c8-d0d2-4dc5-baa7-a9c3d87dff61 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:16.077757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f684c8-d0d2-4dc5-baa7-a9c3d87dff61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:16.078066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f684c8-d0d2-4dc5-baa7-a9c3d87dff61 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:16.078760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f684c8-d0d2-4dc5-baa7-a9c3d87dff61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:42:16.079068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f684c8-d0d2-4dc5-baa7-a9c3d87dff61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:16.080649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f684c8-d0d2-4dc5-baa7-a9c3d87dff61 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:16.092473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1011/2021] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:42:16.096804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1421c7-c8c2-4950-9856-12ee10fc7168 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:16.105778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1421c7-c8c2-4950-9856-12ee10fc7168 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:16.110771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5f619-330a-4373-a88d-ec51ec8aeebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:16.111435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5f619-330a-4373-a88d-ec51ec8aeebc admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:16.111759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5f619-330a-4373-a88d-ec51ec8aeebc admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:16.112102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5f619-330a-4373-a88d-ec51ec8aeebc admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:16.112727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5f619-330a-4373-a88d-ec51ec8aeebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:16.114775 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5f619-330a-4373-a88d-ec51ec8aeebc admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:16.270991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1011/2022] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:16 2022] POST /identity/v3/projects => generated 363 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:16.275198 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f684c8-d0d2-4dc5-baa7-a9c3d87dff61 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:16.285168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f684c8-d0d2-4dc5-baa7-a9c3d87dff61 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:16.310893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:16.342526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8df8b-3b86-49ec-bd94-303ffd983da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:16.343468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8df8b-3b86-49ec-bd94-303ffd983da7 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:16.343814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8df8b-3b86-49ec-bd94-303ffd983da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:16.344089 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8df8b-3b86-49ec-bd94-303ffd983da7 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:16.344793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8df8b-3b86-49ec-bd94-303ffd983da7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:16.346736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8df8b-3b86-49ec-bd94-303ffd983da7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:16.446026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1012/2023] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:16 2022] POST /identity/v3/users => generated 282 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:16.450132 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5f619-330a-4373-a88d-ec51ec8aeebc admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:16.461761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5f619-330a-4373-a88d-ec51ec8aeebc admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:16.467722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f46bc-8551-49e7-8c50-e1048cc3d99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:16.468506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f46bc-8551-49e7-8c50-e1048cc3d99c admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:16.468914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f46bc-8551-49e7-8c50-e1048cc3d99c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:16.469259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f46bc-8551-49e7-8c50-e1048cc3d99c admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:16.470353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f46bc-8551-49e7-8c50-e1048cc3d99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:16.472027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f46bc-8551-49e7-8c50-e1048cc3d99c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:16.485172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1012/2024] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:16.487610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8df8b-3b86-49ec-bd94-303ffd983da7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:16.498104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8df8b-3b86-49ec-bd94-303ffd983da7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:16.502864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1223235-b7e0-45ed-935e-ccd6fd724e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:16.503479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1223235-b7e0-45ed-935e-ccd6fd724e4b admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:16.503762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1223235-b7e0-45ed-935e-ccd6fd724e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:16.504036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1223235-b7e0-45ed-935e-ccd6fd724e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:16.504726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1223235-b7e0-45ed-935e-ccd6fd724e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:16.506381 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1223235-b7e0-45ed-935e-ccd6fd724e4b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:16.517378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1013/2025] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:42:16.521393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f46bc-8551-49e7-8c50-e1048cc3d99c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:16.537424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f46bc-8551-49e7-8c50-e1048cc3d99c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:16.542194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44537917-0b51-40fe-b773-de008c3b5461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:16.542883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44537917-0b51-40fe-b773-de008c3b5461 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:16.543171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44537917-0b51-40fe-b773-de008c3b5461 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:16.543448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44537917-0b51-40fe-b773-de008c3b5461 admin admin] PATH_INFO: `/v3/projects/75158c1c235646848976b2baa1b457bd/users/94a14c0fc19244c8b1c8fab889673444/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:16.559312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44537917-0b51-40fe-b773-de008c3b5461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75158c1c235646848976b2baa1b457bd, user_id=94a14c0fc19244c8b1c8fab889673444, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:16.562095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44537917-0b51-40fe-b773-de008c3b5461 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:16.617028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1013/2026] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:16 2022] PUT /identity/v3/projects/75158c1c235646848976b2baa1b457bd/users/94a14c0fc19244c8b1c8fab889673444/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:42:16.632299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579b3cb-55f3-443d-aabb-9174025f6f16 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:16.632596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579b3cb-55f3-443d-aabb-9174025f6f16 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:16.632873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579b3cb-55f3-443d-aabb-9174025f6f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:16.665524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-9579b3cb-55f3-443d-aabb-9174025f6f16 None None] MFA Rules not processed for user `94a14c0fc19244c8b1c8fab889673444`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:16.666781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-9579b3cb-55f3-443d-aabb-9174025f6f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:16.731822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1014/2027] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:16.737182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44537917-0b51-40fe-b773-de008c3b5461 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:16.799554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44537917-0b51-40fe-b773-de008c3b5461 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:16.804620 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6d6d9-ca1e-4ad8-8558-1721b508031b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:16.805241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6d6d9-ca1e-4ad8-8558-1721b508031b admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:16.805528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6d6d9-ca1e-4ad8-8558-1721b508031b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:16.805807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6d6d9-ca1e-4ad8-8558-1721b508031b admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:16.806540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6d6d9-ca1e-4ad8-8558-1721b508031b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:16.808435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6d6d9-ca1e-4ad8-8558-1721b508031b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:16.884700 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1014/2028] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:16 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:16.888852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579b3cb-55f3-443d-aabb-9174025f6f16 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:16.899380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579b3cb-55f3-443d-aabb-9174025f6f16 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:16.955510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c11a01-e1bb-46db-a931-8cef898d50c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:16.956180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c11a01-e1bb-46db-a931-8cef898d50c2 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:16.956404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c11a01-e1bb-46db-a931-8cef898d50c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:16.956698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c11a01-e1bb-46db-a931-8cef898d50c2 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:16.957361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c11a01-e1bb-46db-a931-8cef898d50c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:16.959295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c11a01-e1bb-46db-a931-8cef898d50c2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:17.061778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1015/2029] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:16 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:17.066197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6d6d9-ca1e-4ad8-8558-1721b508031b admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:17.078028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6d6d9-ca1e-4ad8-8558-1721b508031b admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:17.082868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efe5c0-9f6e-4603-bdcb-8238ddba43ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:17.083486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efe5c0-9f6e-4603-bdcb-8238ddba43ee admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:17.083770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efe5c0-9f6e-4603-bdcb-8238ddba43ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:17.084047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efe5c0-9f6e-4603-bdcb-8238ddba43ee admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:17.084750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efe5c0-9f6e-4603-bdcb-8238ddba43ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:17.086420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efe5c0-9f6e-4603-bdcb-8238ddba43ee admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:17.097694 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1015/2030] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:17.102198 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c11a01-e1bb-46db-a931-8cef898d50c2 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:17.113316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c11a01-e1bb-46db-a931-8cef898d50c2 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:17.118039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e0867-53d9-4599-ba4a-dad2865fdef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:17.118993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e0867-53d9-4599-ba4a-dad2865fdef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:17.119277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e0867-53d9-4599-ba4a-dad2865fdef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:17.119555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e0867-53d9-4599-ba4a-dad2865fdef9 admin admin] PATH_INFO: `/v3/projects/cec4adf8723e480f8d80a87f7d1bf95b/users/5128a102365a4f23a9da4a6f5966f0e8/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:17.134124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e0867-53d9-4599-ba4a-dad2865fdef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec4adf8723e480f8d80a87f7d1bf95b, user_id=5128a102365a4f23a9da4a6f5966f0e8, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:17.136822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e0867-53d9-4599-ba4a-dad2865fdef9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:17.206384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1016/2031] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:17 2022] PUT /identity/v3/projects/cec4adf8723e480f8d80a87f7d1bf95b/users/5128a102365a4f23a9da4a6f5966f0e8/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:17.214087 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efe5c0-9f6e-4603-bdcb-8238ddba43ee admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:17.224917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efe5c0-9f6e-4603-bdcb-8238ddba43ee admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:17.246465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:17.287918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2807d-33a8-48ea-a84e-119f2a153903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:17.288569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2807d-33a8-48ea-a84e-119f2a153903 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:17.288849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2807d-33a8-48ea-a84e-119f2a153903 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:17.289122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2807d-33a8-48ea-a84e-119f2a153903 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:17.289863 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2807d-33a8-48ea-a84e-119f2a153903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:17.291574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2807d-33a8-48ea-a84e-119f2a153903 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:17.305500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1016/2032] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:17.306658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e0867-53d9-4599-ba4a-dad2865fdef9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:17.312428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:17.317825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e0867-53d9-4599-ba4a-dad2865fdef9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:17.322456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89b634-e8dc-4f8d-8a51-7cc9d5713fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:17.323758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89b634-e8dc-4f8d-8a51-7cc9d5713fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:17.324038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89b634-e8dc-4f8d-8a51-7cc9d5713fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:17.324311 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89b634-e8dc-4f8d-8a51-7cc9d5713fe6 admin admin] PATH_INFO: `/v3/projects/cec4adf8723e480f8d80a87f7d1bf95b/users/5128a102365a4f23a9da4a6f5966f0e8/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:17.326446 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89b634-e8dc-4f8d-8a51-7cc9d5713fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec4adf8723e480f8d80a87f7d1bf95b, user_id=5128a102365a4f23a9da4a6f5966f0e8, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:17.329098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89b634-e8dc-4f8d-8a51-7cc9d5713fe6 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:17.339507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1017/2033] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:17 2022] PUT /identity/v3/projects/cec4adf8723e480f8d80a87f7d1bf95b/users/5128a102365a4f23a9da4a6f5966f0e8/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:42:17.351250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc89f2-336c-40a0-8e3f-aa578a99aed2 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:17.351539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc89f2-336c-40a0-8e3f-aa578a99aed2 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:17.351815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc89f2-336c-40a0-8e3f-aa578a99aed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:17.380229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-5fcc89f2-336c-40a0-8e3f-aa578a99aed2 None None] MFA Rules not processed for user `5128a102365a4f23a9da4a6f5966f0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:17.381439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5fcc89f2-336c-40a0-8e3f-aa578a99aed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:17.454921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1017/2034] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:17.466197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89b634-e8dc-4f8d-8a51-7cc9d5713fe6 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:17.468654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-9d89b634-e8dc-4f8d-8a51-7cc9d5713fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:17.524554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89b634-e8dc-4f8d-8a51-7cc9d5713fe6 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:17.528180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dc35c-72ed-4c68-8b45-a41e72397890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:17.528798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dc35c-72ed-4c68-8b45-a41e72397890 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:17.529078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dc35c-72ed-4c68-8b45-a41e72397890 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:17.529430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dc35c-72ed-4c68-8b45-a41e72397890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:17.538784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09dc35c-72ed-4c68-8b45-a41e72397890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:17.540196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09dc35c-72ed-4c68-8b45-a41e72397890 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:17.550304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1018/2035] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:17.768934 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc89f2-336c-40a0-8e3f-aa578a99aed2 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:17.779012 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc89f2-336c-40a0-8e3f-aa578a99aed2 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:17.783468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8453ee3-c831-4fa7-956f-b423c93398ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:17.784082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8453ee3-c831-4fa7-956f-b423c93398ae service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:17.784361 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8453ee3-c831-4fa7-956f-b423c93398ae service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:17.784630 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8453ee3-c831-4fa7-956f-b423c93398ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:17.840345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8453ee3-c831-4fa7-956f-b423c93398ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:17.842396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8453ee3-c831-4fa7-956f-b423c93398ae service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:17.851312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1018/2036] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:21.396680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dc35c-72ed-4c68-8b45-a41e72397890 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:21.463140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dc35c-72ed-4c68-8b45-a41e72397890 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:21.467711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:21.469025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:21.469323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:21.469581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] PATH_INFO: `/v3/users/94a14c0fc19244c8b1c8fab889673444` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:21.471083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94a14c0fc19244c8b1c8fab889673444)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:21.473047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:21.670234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94a14c0fc19244c8b1c8fab889673444'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:21.734843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94a14c0fc19244c8b1c8fab889673444'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:21.747377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94a14c0fc19244c8b1c8fab889673444'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:21.748043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94a14c0fc19244c8b1c8fab889673444'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:21.755375 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94a14c0fc19244c8b1c8fab889673444'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:21.761230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1019/2037] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:21 2022] DELETE /identity/v3/users/94a14c0fc19244c8b1c8fab889673444 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:21.766492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0909f2-40d7-43e0-b6c9-e0e331ce43b3 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:21.766953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0909f2-40d7-43e0-b6c9-e0e331ce43b3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:21.767078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0909f2-40d7-43e0-b6c9-e0e331ce43b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:21.786458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-5f0909f2-40d7-43e0-b6c9-e0e331ce43b3 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:21.787672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5f0909f2-40d7-43e0-b6c9-e0e331ce43b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:21.828516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:21.847070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1019/2038] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:21.857462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:21.859508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:21.878986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:21.924452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54677a96-9a06-4c5a-8fab-a17b07beb314 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:21.928129 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86ca5c-52f9-441c-98df-78fdd882ba7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:21.928753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86ca5c-52f9-441c-98df-78fdd882ba7d service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:21.929038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86ca5c-52f9-441c-98df-78fdd882ba7d service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:21.929314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86ca5c-52f9-441c-98df-78fdd882ba7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:21.940629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86ca5c-52f9-441c-98df-78fdd882ba7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:21.942331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86ca5c-52f9-441c-98df-78fdd882ba7d service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:21.952484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1020/2039] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:22.350913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0909f2-40d7-43e0-b6c9-e0e331ce43b3 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:22.352433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5f0909f2-40d7-43e0-b6c9-e0e331ce43b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:22.365173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0909f2-40d7-43e0-b6c9-e0e331ce43b3 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:22.370109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:22.370750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:22.371039 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:22.371311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] PATH_INFO: `/v3/projects/75158c1c235646848976b2baa1b457bd` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:22.372500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75158c1c235646848976b2baa1b457bd)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:22.374632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:22.502124 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75158c1c235646848976b2baa1b457bd', 'request_id': 'req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:22.510825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75158c1c235646848976b2baa1b457bd', 'request_id': 'req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:22.588457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75158c1c235646848976b2baa1b457bd', 'request_id': 'req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:22.601726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] The token cache is being invalidate because project 75158c1c235646848976b2baa1b457bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:22.602497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:22.608165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:22.613654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1020/2040] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:22 2022] DELETE /identity/v3/projects/75158c1c235646848976b2baa1b457bd => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:22.616527 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86ca5c-52f9-441c-98df-78fdd882ba7d service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:22.630797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86ca5c-52f9-441c-98df-78fdd882ba7d service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:22.690155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de7958-ba67-4aa9-a6c3-7ee7920714c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:22.690800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de7958-ba67-4aa9-a6c3-7ee7920714c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:22.696732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de7958-ba67-4aa9-a6c3-7ee7920714c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:22.696732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de7958-ba67-4aa9-a6c3-7ee7920714c7 admin admin] PATH_INFO: `/v3/users/5128a102365a4f23a9da4a6f5966f0e8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:22.696732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de7958-ba67-4aa9-a6c3-7ee7920714c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5128a102365a4f23a9da4a6f5966f0e8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:22.696732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de7958-ba67-4aa9-a6c3-7ee7920714c7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:22.881611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-48de7958-ba67-4aa9-a6c3-7ee7920714c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5128a102365a4f23a9da4a6f5966f0e8'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:22.930685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-48de7958-ba67-4aa9-a6c3-7ee7920714c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5128a102365a4f23a9da4a6f5966f0e8'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:22.943487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-48de7958-ba67-4aa9-a6c3-7ee7920714c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5128a102365a4f23a9da4a6f5966f0e8'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:22.944143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-48de7958-ba67-4aa9-a6c3-7ee7920714c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5128a102365a4f23a9da4a6f5966f0e8'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:22.950934 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-48de7958-ba67-4aa9-a6c3-7ee7920714c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5128a102365a4f23a9da4a6f5966f0e8'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:22.957498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1021/2041] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:22 2022] DELETE /identity/v3/users/5128a102365a4f23a9da4a6f5966f0e8 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:23.361591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:23.363125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:23.378607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1dfb6-8eeb-4f1d-96e8-b3a56505cf45 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:23.443989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:23.444852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:23.445145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:23.445424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] PATH_INFO: `/v3/projects/cec4adf8723e480f8d80a87f7d1bf95b` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:23.446627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec4adf8723e480f8d80a87f7d1bf95b)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:23.448581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:23.553139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec4adf8723e480f8d80a87f7d1bf95b', 'request_id': 'req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:23.559404 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec4adf8723e480f8d80a87f7d1bf95b', 'request_id': 'req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:23.666311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec4adf8723e480f8d80a87f7d1bf95b', 'request_id': 'req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:23.667387 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:23.668764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] The token cache is being invalidate because project cec4adf8723e480f8d80a87f7d1bf95b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:23.669202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:23.669930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-fdd98bbb-37b2-484e-b1d7-e859204d9e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:23.672281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1021/2042] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:23 2022] DELETE /identity/v3/projects/cec4adf8723e480f8d80a87f7d1bf95b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:23.680397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0fdaf-5579-48e7-982f-d9b403b127cf None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:23.680918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0fdaf-5579-48e7-982f-d9b403b127cf None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:23.681055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0fdaf-5579-48e7-982f-d9b403b127cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:23.703823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-93d0fdaf-5579-48e7-982f-d9b403b127cf None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:23.705298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-93d0fdaf-5579-48e7-982f-d9b403b127cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:23.737732 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:23.767734 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1022/2043] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:23.775888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969b9e-07d2-433d-8813-170813bb2864 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:23.776370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969b9e-07d2-433d-8813-170813bb2864 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:23.776645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969b9e-07d2-433d-8813-170813bb2864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:23.799130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a3969b9e-07d2-433d-8813-170813bb2864 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:23.800355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a3969b9e-07d2-433d-8813-170813bb2864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:23.809364 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1022/2044] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:42:23.813743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0fdaf-5579-48e7-982f-d9b403b127cf None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:23.823456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0fdaf-5579-48e7-982f-d9b403b127cf None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:23.828861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd07d2f-fa95-44c6-a6df-0229bda1b7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:23.829684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd07d2f-fa95-44c6-a6df-0229bda1b7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:23.830067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd07d2f-fa95-44c6-a6df-0229bda1b7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:23.830407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd07d2f-fa95-44c6-a6df-0229bda1b7e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:23.831132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd07d2f-fa95-44c6-a6df-0229bda1b7e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:42:23.831439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd07d2f-fa95-44c6-a6df-0229bda1b7e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:23.833017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd07d2f-fa95-44c6-a6df-0229bda1b7e8 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:23.845768 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1023/2045] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:42:23.850359 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3969b9e-07d2-433d-8813-170813bb2864 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:23.860188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3969b9e-07d2-433d-8813-170813bb2864 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:23.864887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c9426-484e-465c-891c-8a091046346b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:23.865484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c9426-484e-465c-891c-8a091046346b admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:23.865759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c9426-484e-465c-891c-8a091046346b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:23.866054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c9426-484e-465c-891c-8a091046346b admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:23.866723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c9426-484e-465c-891c-8a091046346b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:23.868884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c9426-484e-465c-891c-8a091046346b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:23.971251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1023/2046] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:23 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:23.975456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd07d2f-fa95-44c6-a6df-0229bda1b7e8 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:23.985872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd07d2f-fa95-44c6-a6df-0229bda1b7e8 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:24.043363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00c23f-e018-4a72-aa8b-8ef5b1bfa1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:24.044825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c23f-e018-4a72-aa8b-8ef5b1bfa1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:24.045189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c23f-e018-4a72-aa8b-8ef5b1bfa1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:24.045460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c23f-e018-4a72-aa8b-8ef5b1bfa1b4 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:24.046154 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00c23f-e018-4a72-aa8b-8ef5b1bfa1b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:24.048775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00c23f-e018-4a72-aa8b-8ef5b1bfa1b4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:24.148648 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1024/2047] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:23 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:24.152928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c9426-484e-465c-891c-8a091046346b admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:24.163799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c9426-484e-465c-891c-8a091046346b admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:24.168426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3de71d-36e4-40e2-a0e3-d104e98d7de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:24.169034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3de71d-36e4-40e2-a0e3-d104e98d7de2 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:24.169319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3de71d-36e4-40e2-a0e3-d104e98d7de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:24.169613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3de71d-36e4-40e2-a0e3-d104e98d7de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:24.170316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3de71d-36e4-40e2-a0e3-d104e98d7de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:24.172109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3de71d-36e4-40e2-a0e3-d104e98d7de2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:24.181915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1024/2048] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:42:24.186103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00c23f-e018-4a72-aa8b-8ef5b1bfa1b4 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:24.196206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00c23f-e018-4a72-aa8b-8ef5b1bfa1b4 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:24.200882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beadaf5-a5fe-4350-882f-f6c8e5dbb3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:24.201479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadaf5-a5fe-4350-882f-f6c8e5dbb3fc admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:24.201762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadaf5-a5fe-4350-882f-f6c8e5dbb3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:24.202068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadaf5-a5fe-4350-882f-f6c8e5dbb3fc admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:24.203322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beadaf5-a5fe-4350-882f-f6c8e5dbb3fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:24.205114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beadaf5-a5fe-4350-882f-f6c8e5dbb3fc admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:24.215767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1025/2049] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:24.220629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3de71d-36e4-40e2-a0e3-d104e98d7de2 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:24.230742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3de71d-36e4-40e2-a0e3-d104e98d7de2 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:24.235292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6ec57-865b-48f4-872d-075d160bd1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:24.235947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6ec57-865b-48f4-872d-075d160bd1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:24.236231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6ec57-865b-48f4-872d-075d160bd1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:24.236506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6ec57-865b-48f4-872d-075d160bd1cf admin admin] PATH_INFO: `/v3/projects/1780736dca254657bbcfec379ba1e5f8/users/0f4b2088e2d147b4933fd9fd948b0cb6/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:24.251303 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6ec57-865b-48f4-872d-075d160bd1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1780736dca254657bbcfec379ba1e5f8, user_id=0f4b2088e2d147b4933fd9fd948b0cb6, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:24.253962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6ec57-865b-48f4-872d-075d160bd1cf admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:24.354469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1025/2050] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:24 2022] PUT /identity/v3/projects/1780736dca254657bbcfec379ba1e5f8/users/0f4b2088e2d147b4933fd9fd948b0cb6/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:24.364942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb976f-d1c6-4a68-b557-fe540126ead1 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:24.365235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb976f-d1c6-4a68-b557-fe540126ead1 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:24.365514 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb976f-d1c6-4a68-b557-fe540126ead1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:24.395281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-50bb976f-d1c6-4a68-b557-fe540126ead1 None None] MFA Rules not processed for user `0f4b2088e2d147b4933fd9fd948b0cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:24.397398 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-50bb976f-d1c6-4a68-b557-fe540126ead1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:24.464961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1026/2051] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:24.469709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6ec57-865b-48f4-872d-075d160bd1cf admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:24.532592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6ec57-865b-48f4-872d-075d160bd1cf admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:24.537421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e3b0f-4512-428c-b9cd-e934adbf0b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:24.538059 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e3b0f-4512-428c-b9cd-e934adbf0b04 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:24.539193 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e3b0f-4512-428c-b9cd-e934adbf0b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:24.539494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e3b0f-4512-428c-b9cd-e934adbf0b04 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:24.540313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e3b0f-4512-428c-b9cd-e934adbf0b04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:24.543207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e3b0f-4512-428c-b9cd-e934adbf0b04 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:24.629214 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1026/2052] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:24 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:24.632746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb976f-d1c6-4a68-b557-fe540126ead1 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:24.644166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb976f-d1c6-4a68-b557-fe540126ead1 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:24.669748 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:24.710930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b00a74-c769-44fb-bfc2-928b35257ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:24.711561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b00a74-c769-44fb-bfc2-928b35257ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:24.711906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b00a74-c769-44fb-bfc2-928b35257ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:24.712125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b00a74-c769-44fb-bfc2-928b35257ef0 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:24.712791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b00a74-c769-44fb-bfc2-928b35257ef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:24.714762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b00a74-c769-44fb-bfc2-928b35257ef0 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:24.801935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1027/2053] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:24 2022] POST /identity/v3/users => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:24.803056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e3b0f-4512-428c-b9cd-e934adbf0b04 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:24.814259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e3b0f-4512-428c-b9cd-e934adbf0b04 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:24.819185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65f1b5-fb75-4220-9523-e08445884f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:24.819800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65f1b5-fb75-4220-9523-e08445884f14 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:24.820085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65f1b5-fb75-4220-9523-e08445884f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:24.820386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65f1b5-fb75-4220-9523-e08445884f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:24.821081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65f1b5-fb75-4220-9523-e08445884f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:24.822751 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65f1b5-fb75-4220-9523-e08445884f14 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:24.836019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1027/2054] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:42:24.839552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b00a74-c769-44fb-bfc2-928b35257ef0 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:24.850470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b00a74-c769-44fb-bfc2-928b35257ef0 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:24.855330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bc874-13a1-4375-b631-62d873f28ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:24.856084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bc874-13a1-4375-b631-62d873f28ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:24.856369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bc874-13a1-4375-b631-62d873f28ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:24.856664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bc874-13a1-4375-b631-62d873f28ecb admin admin] PATH_INFO: `/v3/projects/0a2a201fa2164e908f408bb07879b7cd/users/42b2524abf404ffe94aa5fc1b095e62d/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:24.873828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bc874-13a1-4375-b631-62d873f28ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2a201fa2164e908f408bb07879b7cd, user_id=42b2524abf404ffe94aa5fc1b095e62d, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:24.876744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bc874-13a1-4375-b631-62d873f28ecb admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:24.931789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1028/2055] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:24 2022] PUT /identity/v3/projects/0a2a201fa2164e908f408bb07879b7cd/users/42b2524abf404ffe94aa5fc1b095e62d/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:24.935809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65f1b5-fb75-4220-9523-e08445884f14 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:24.946443 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65f1b5-fb75-4220-9523-e08445884f14 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:25.006891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13224940-672c-4439-9268-238a77918e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:25.007589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13224940-672c-4439-9268-238a77918e52 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:25.007947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13224940-672c-4439-9268-238a77918e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:25.008283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13224940-672c-4439-9268-238a77918e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:25.008978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13224940-672c-4439-9268-238a77918e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:25.010834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13224940-672c-4439-9268-238a77918e52 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:25.025996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1028/2056] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:42:25.030224 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bc874-13a1-4375-b631-62d873f28ecb admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:25.041582 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bc874-13a1-4375-b631-62d873f28ecb admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:25.046277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0245ab-dadb-49a5-a7fe-4754d71e23ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:25.046955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0245ab-dadb-49a5-a7fe-4754d71e23ba admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:25.047394 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0245ab-dadb-49a5-a7fe-4754d71e23ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:25.047672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0245ab-dadb-49a5-a7fe-4754d71e23ba admin admin] PATH_INFO: `/v3/projects/0a2a201fa2164e908f408bb07879b7cd/users/42b2524abf404ffe94aa5fc1b095e62d/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:25.049631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0245ab-dadb-49a5-a7fe-4754d71e23ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2a201fa2164e908f408bb07879b7cd, user_id=42b2524abf404ffe94aa5fc1b095e62d, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:25.052339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0245ab-dadb-49a5-a7fe-4754d71e23ba admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:25.062636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1029/2057] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:25 2022] PUT /identity/v3/projects/0a2a201fa2164e908f408bb07879b7cd/users/42b2524abf404ffe94aa5fc1b095e62d/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:25.074511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06134939-77ef-4363-8227-36727c3bfc46 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:25.074799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06134939-77ef-4363-8227-36727c3bfc46 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:25.075226 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06134939-77ef-4363-8227-36727c3bfc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:25.084258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:25.104021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-06134939-77ef-4363-8227-36727c3bfc46 None None] MFA Rules not processed for user `42b2524abf404ffe94aa5fc1b095e62d`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:25.105234 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-06134939-77ef-4363-8227-36727c3bfc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:25.172849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1029/2058] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:25.184206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0245ab-dadb-49a5-a7fe-4754d71e23ba admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:25.186317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4e0245ab-dadb-49a5-a7fe-4754d71e23ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:25.244452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0245ab-dadb-49a5-a7fe-4754d71e23ba admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:25.248091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857346c8-24f9-4e6e-8dfd-d42be47bb5fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:25.248700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857346c8-24f9-4e6e-8dfd-d42be47bb5fc service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:25.248978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857346c8-24f9-4e6e-8dfd-d42be47bb5fc service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:25.249249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857346c8-24f9-4e6e-8dfd-d42be47bb5fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:25.259123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857346c8-24f9-4e6e-8dfd-d42be47bb5fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:25.260610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857346c8-24f9-4e6e-8dfd-d42be47bb5fc service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:25.270830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1030/2059] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:25.376577 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06134939-77ef-4363-8227-36727c3bfc46 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:25.386371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06134939-77ef-4363-8227-36727c3bfc46 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:25.389922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ffd1f-c349-4a3e-9fc5-6ccc19b680ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:25.391241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ffd1f-c349-4a3e-9fc5-6ccc19b680ff service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:25.391634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ffd1f-c349-4a3e-9fc5-6ccc19b680ff service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:25.391919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ffd1f-c349-4a3e-9fc5-6ccc19b680ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:25.449532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ffd1f-c349-4a3e-9fc5-6ccc19b680ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:25.451129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ffd1f-c349-4a3e-9fc5-6ccc19b680ff service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:25.460788 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1030/2060] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:25.621966 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857346c8-24f9-4e6e-8dfd-d42be47bb5fc service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:25.623521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-857346c8-24f9-4e6e-8dfd-d42be47bb5fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:25.690433 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857346c8-24f9-4e6e-8dfd-d42be47bb5fc service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:25.695412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:25.696045 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:25.696345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:25.696617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] PATH_INFO: `/v3/users/adb73f91cf1d482e8c43588ad13bedae` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:25.697808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb73f91cf1d482e8c43588ad13bedae)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:25.699750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:25.821684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:25.916505 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb73f91cf1d482e8c43588ad13bedae'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:25.984538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb73f91cf1d482e8c43588ad13bedae'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:26.009753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb73f91cf1d482e8c43588ad13bedae'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:26.010928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb73f91cf1d482e8c43588ad13bedae'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:26.019284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb73f91cf1d482e8c43588ad13bedae'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:26.025410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1031/2061] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:25 2022] DELETE /identity/v3/users/adb73f91cf1d482e8c43588ad13bedae => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:26.032482 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c288c8-00bd-42f1-8327-9e4537e81402 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:26.032987 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c288c8-00bd-42f1-8327-9e4537e81402 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:26.033165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c288c8-00bd-42f1-8327-9e4537e81402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:26.059752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-42c288c8-00bd-42f1-8327-9e4537e81402 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:26.060696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-42c288c8-00bd-42f1-8327-9e4537e81402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:26.140095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1031/2062] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:26.150264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:26.151675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:26.213980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78811b9-c373-4a97-8085-85d708e2e348 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:26.217792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1b017-34c4-4dd0-aead-b6b3e96f7d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:26.219100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1b017-34c4-4dd0-aead-b6b3e96f7d5e service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:26.220126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1b017-34c4-4dd0-aead-b6b3e96f7d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:26.220546 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1b017-34c4-4dd0-aead-b6b3e96f7d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:26.230229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1b017-34c4-4dd0-aead-b6b3e96f7d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:26.231650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1b017-34c4-4dd0-aead-b6b3e96f7d5e service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:26.241537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1032/2063] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:26.646984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c288c8-00bd-42f1-8327-9e4537e81402 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:26.648422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-42c288c8-00bd-42f1-8327-9e4537e81402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:26.660525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c288c8-00bd-42f1-8327-9e4537e81402 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:26.665566 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:26.666226 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:26.666506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:26.666774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] PATH_INFO: `/v3/projects/fd9047156c9d49c2a5e1280cc28d9a16` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:26.667927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9047156c9d49c2a5e1280cc28d9a16)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:26.669849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:26.696164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:26.811881 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9047156c9d49c2a5e1280cc28d9a16', 'request_id': 'req-b03e9413-4c00-4ca2-8872-318f1c77d6be'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:26.818200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9047156c9d49c2a5e1280cc28d9a16', 'request_id': 'req-b03e9413-4c00-4ca2-8872-318f1c77d6be'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:26.913789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9047156c9d49c2a5e1280cc28d9a16', 'request_id': 'req-b03e9413-4c00-4ca2-8872-318f1c77d6be'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:26.925659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] The token cache is being invalidate because project fd9047156c9d49c2a5e1280cc28d9a16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:26.925948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:26.934526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-b03e9413-4c00-4ca2-8872-318f1c77d6be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:26.934526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1032/2064] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:26 2022] DELETE /identity/v3/projects/fd9047156c9d49c2a5e1280cc28d9a16 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:26.934713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57c179-a249-46c8-b432-d723812bfc23 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:26.934713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57c179-a249-46c8-b432-d723812bfc23 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:26.934713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57c179-a249-46c8-b432-d723812bfc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:26.952440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-de57c179-a249-46c8-b432-d723812bfc23 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:26.953781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-de57c179-a249-46c8-b432-d723812bfc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:27.010762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1033/2065] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:27.018877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87a50e-ce44-4631-ac05-ef6fc6fb9952 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:27.019169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87a50e-ce44-4631-ac05-ef6fc6fb9952 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:27.019478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87a50e-ce44-4631-ac05-ef6fc6fb9952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:27.041646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1d87a50e-ce44-4631-ac05-ef6fc6fb9952 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:27.042918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1d87a50e-ce44-4631-ac05-ef6fc6fb9952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:27.052299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1033/2066] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:27.056702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57c179-a249-46c8-b432-d723812bfc23 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:27.067085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57c179-a249-46c8-b432-d723812bfc23 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:27.071899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7db22-8c70-48bc-8d94-fc39baa97a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:27.072511 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7db22-8c70-48bc-8d94-fc39baa97a30 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:27.072789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7db22-8c70-48bc-8d94-fc39baa97a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:27.073058 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7db22-8c70-48bc-8d94-fc39baa97a30 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:27.073780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7db22-8c70-48bc-8d94-fc39baa97a30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:42:27.074108 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7db22-8c70-48bc-8d94-fc39baa97a30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:27.075699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7db22-8c70-48bc-8d94-fc39baa97a30 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:27.089370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1034/2067] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:42:27.093546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87a50e-ce44-4631-ac05-ef6fc6fb9952 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:27.103741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87a50e-ce44-4631-ac05-ef6fc6fb9952 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:27.108635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad18c63-5bf2-4b14-bd81-ff1bf4ef30ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:27.109322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad18c63-5bf2-4b14-bd81-ff1bf4ef30ce admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:27.109736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad18c63-5bf2-4b14-bd81-ff1bf4ef30ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:27.110364 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad18c63-5bf2-4b14-bd81-ff1bf4ef30ce admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:27.111099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad18c63-5bf2-4b14-bd81-ff1bf4ef30ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:27.113064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad18c63-5bf2-4b14-bd81-ff1bf4ef30ce admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:27.201486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1034/2068] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:27 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:27.207259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7db22-8c70-48bc-8d94-fc39baa97a30 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:27.222104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7db22-8c70-48bc-8d94-fc39baa97a30 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:27.255922 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:27.283412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bb650-0967-4bef-b0d7-a7f1698a82ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:27.284033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb650-0967-4bef-b0d7-a7f1698a82ad admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:27.284313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb650-0967-4bef-b0d7-a7f1698a82ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:27.284584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb650-0967-4bef-b0d7-a7f1698a82ad admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:27.285241 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bb650-0967-4bef-b0d7-a7f1698a82ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:27.287212 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bb650-0967-4bef-b0d7-a7f1698a82ad admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:27.363584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1035/2069] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:27 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:27.364726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad18c63-5bf2-4b14-bd81-ff1bf4ef30ce admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:27.375952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad18c63-5bf2-4b14-bd81-ff1bf4ef30ce admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:27.381005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6c03c-4ade-4df5-a960-e2830cda3689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:27.381671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6c03c-4ade-4df5-a960-e2830cda3689 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:27.381953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6c03c-4ade-4df5-a960-e2830cda3689 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:27.389004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6c03c-4ade-4df5-a960-e2830cda3689 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:27.389004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6c03c-4ade-4df5-a960-e2830cda3689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:27.389004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6c03c-4ade-4df5-a960-e2830cda3689 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:27.395062 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1035/2070] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:27.399194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bb650-0967-4bef-b0d7-a7f1698a82ad admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:27.412124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bb650-0967-4bef-b0d7-a7f1698a82ad admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:27.416747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b73915-0903-46ec-98ce-c48aeee9108d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:27.417341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73915-0903-46ec-98ce-c48aeee9108d admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:27.417636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73915-0903-46ec-98ce-c48aeee9108d admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:27.417889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73915-0903-46ec-98ce-c48aeee9108d admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:27.418614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b73915-0903-46ec-98ce-c48aeee9108d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:27.420174 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b73915-0903-46ec-98ce-c48aeee9108d admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:27.430916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1036/2071] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:27.435568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6c03c-4ade-4df5-a960-e2830cda3689 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:27.446429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6c03c-4ade-4df5-a960-e2830cda3689 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:27.451016 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a3d40-29c0-4d32-8882-19a4df55a5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:27.451664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a3d40-29c0-4d32-8882-19a4df55a5b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:27.451945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a3d40-29c0-4d32-8882-19a4df55a5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:27.452220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a3d40-29c0-4d32-8882-19a4df55a5b2 admin admin] PATH_INFO: `/v3/projects/f738ce353ad649709334f71983434d3c/users/7487daf672c04b1bb052a8a5e4875d61/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:27.467878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a3d40-29c0-4d32-8882-19a4df55a5b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f738ce353ad649709334f71983434d3c, user_id=7487daf672c04b1bb052a8a5e4875d61, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:27.471142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a3d40-29c0-4d32-8882-19a4df55a5b2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:27.553879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1036/2072] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:27 2022] PUT /identity/v3/projects/f738ce353ad649709334f71983434d3c/users/7487daf672c04b1bb052a8a5e4875d61/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:27.564775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb333dbf-e0aa-4eb6-a7fe-ab84ec8de731 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:27.565065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb333dbf-e0aa-4eb6-a7fe-ab84ec8de731 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:27.565342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb333dbf-e0aa-4eb6-a7fe-ab84ec8de731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:27.594379 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-fb333dbf-e0aa-4eb6-a7fe-ab84ec8de731 None None] MFA Rules not processed for user `7487daf672c04b1bb052a8a5e4875d61`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:27.595638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-fb333dbf-e0aa-4eb6-a7fe-ab84ec8de731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:27.659012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1037/2073] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:27.669466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a3d40-29c0-4d32-8882-19a4df55a5b2 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:27.671191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-118a3d40-29c0-4d32-8882-19a4df55a5b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:27.726981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a3d40-29c0-4d32-8882-19a4df55a5b2 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:27.730514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05505c3-2fd7-4d94-aefa-70e1be351ed7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:27.731118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05505c3-2fd7-4d94-aefa-70e1be351ed7 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:27.731397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05505c3-2fd7-4d94-aefa-70e1be351ed7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:27.731667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05505c3-2fd7-4d94-aefa-70e1be351ed7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:27.739903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05505c3-2fd7-4d94-aefa-70e1be351ed7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:27.741321 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05505c3-2fd7-4d94-aefa-70e1be351ed7 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:27.750633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1037/2074] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:29.577524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb333dbf-e0aa-4eb6-a7fe-ab84ec8de731 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:29.579904 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-fb333dbf-e0aa-4eb6-a7fe-ab84ec8de731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:29.650749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb333dbf-e0aa-4eb6-a7fe-ab84ec8de731 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:29.655276 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:29.657108 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:29.657474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:29.657758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] PATH_INFO: `/v3/users/0f4b2088e2d147b4933fd9fd948b0cb6` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:29.659039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f4b2088e2d147b4933fd9fd948b0cb6)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:29.661144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:29.810113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:29.813084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f4b2088e2d147b4933fd9fd948b0cb6'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:29.895448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f4b2088e2d147b4933fd9fd948b0cb6'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:29.907759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f4b2088e2d147b4933fd9fd948b0cb6'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:29.908331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f4b2088e2d147b4933fd9fd948b0cb6'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:29.915107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f4b2088e2d147b4933fd9fd948b0cb6'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:29.920858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1038/2075] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:29 2022] DELETE /identity/v3/users/0f4b2088e2d147b4933fd9fd948b0cb6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:29.931175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bc1c7-6729-4110-9cf2-d7978d0d4132 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:29.931664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bc1c7-6729-4110-9cf2-d7978d0d4132 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:29.931758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bc1c7-6729-4110-9cf2-d7978d0d4132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:29.951158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ea8bc1c7-6729-4110-9cf2-d7978d0d4132 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:29.952392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ea8bc1c7-6729-4110-9cf2-d7978d0d4132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:29.999674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:30.015827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1038/2076] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:30.030699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:30.032900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:30.088728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3bcc2-140f-4936-b286-8551860107af admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:30.092353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8043f-d61c-40e0-b162-75d27d53bf84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:30.092960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8043f-d61c-40e0-b162-75d27d53bf84 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:30.093240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8043f-d61c-40e0-b162-75d27d53bf84 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:30.093505 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8043f-d61c-40e0-b162-75d27d53bf84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:30.103280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8043f-d61c-40e0-b162-75d27d53bf84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:30.104684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8043f-d61c-40e0-b162-75d27d53bf84 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:30.114823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1039/2077] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:30.552485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8520a-2422-4729-8e7b-1b7714844834 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:30.552849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8520a-2422-4729-8e7b-1b7714844834 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:30.553130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8520a-2422-4729-8e7b-1b7714844834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:30.567640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8043f-d61c-40e0-b162-75d27d53bf84 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:30.570263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a6d8043f-d61c-40e0-b162-75d27d53bf84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:30.581973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-33a8520a-2422-4729-8e7b-1b7714844834 None None] MFA Rules not processed for user `7487daf672c04b1bb052a8a5e4875d61`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:30.583196 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-33a8520a-2422-4729-8e7b-1b7714844834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:30.591189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8043f-d61c-40e0-b162-75d27d53bf84 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:30.608355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:30.609310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:30.609600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:30.609997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] PATH_INFO: `/v3/projects/1780736dca254657bbcfec379ba1e5f8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:30.612416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] RBAC: Authorizing `identity:delete_project(project_id=1780736dca254657bbcfec379ba1e5f8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:30.614348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:30.664202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1039/2078] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:30.741756 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1780736dca254657bbcfec379ba1e5f8', 'request_id': 'req-8d44abe9-5007-4f0f-b58a-7c6b25d763de'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:30.823034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1780736dca254657bbcfec379ba1e5f8', 'request_id': 'req-8d44abe9-5007-4f0f-b58a-7c6b25d763de'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:30.832814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1780736dca254657bbcfec379ba1e5f8', 'request_id': 'req-8d44abe9-5007-4f0f-b58a-7c6b25d763de'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:30.851665 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] The token cache is being invalidate because project 1780736dca254657bbcfec379ba1e5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:30.852219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:30.852894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:30.854773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1040/2079] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:30 2022] DELETE /identity/v3/projects/1780736dca254657bbcfec379ba1e5f8 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:42:30.858725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8520a-2422-4729-8e7b-1b7714844834 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:30.870766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8520a-2422-4729-8e7b-1b7714844834 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:30.939232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8052e-647c-47dd-8c77-459cce2ed52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:30.940847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8052e-647c-47dd-8c77-459cce2ed52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:30.940847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8052e-647c-47dd-8c77-459cce2ed52c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:30.940847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8052e-647c-47dd-8c77-459cce2ed52c admin admin] PATH_INFO: `/v3/users/42b2524abf404ffe94aa5fc1b095e62d` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:30.942339 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8052e-647c-47dd-8c77-459cce2ed52c admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b2524abf404ffe94aa5fc1b095e62d)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:30.944454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8052e-647c-47dd-8c77-459cce2ed52c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:31.111239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d0c8052e-647c-47dd-8c77-459cce2ed52c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b2524abf404ffe94aa5fc1b095e62d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:31.176664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d0c8052e-647c-47dd-8c77-459cce2ed52c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b2524abf404ffe94aa5fc1b095e62d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:31.177307 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d0c8052e-647c-47dd-8c77-459cce2ed52c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b2524abf404ffe94aa5fc1b095e62d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:31.192171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d0c8052e-647c-47dd-8c77-459cce2ed52c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b2524abf404ffe94aa5fc1b095e62d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:31.199365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d0c8052e-647c-47dd-8c77-459cce2ed52c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b2524abf404ffe94aa5fc1b095e62d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:31.206131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1040/2080] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:30 2022] DELETE /identity/v3/users/42b2524abf404ffe94aa5fc1b095e62d => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:31.687743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:31.689949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:31.704943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44abe9-5007-4f0f-b58a-7c6b25d763de admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:31.752936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:31.768082 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:31.768781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:31.769066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:31.769342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] PATH_INFO: `/v3/projects/0a2a201fa2164e908f408bb07879b7cd` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:31.770605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2a201fa2164e908f408bb07879b7cd)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:31.772856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:31.915742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2a201fa2164e908f408bb07879b7cd', 'request_id': 'req-96974add-6ddb-4380-ba19-fd75a5eac867'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:31.988217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2a201fa2164e908f408bb07879b7cd', 'request_id': 'req-96974add-6ddb-4380-ba19-fd75a5eac867'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:31.997211 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2a201fa2164e908f408bb07879b7cd', 'request_id': 'req-96974add-6ddb-4380-ba19-fd75a5eac867'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:32.011826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] The token cache is being invalidate because project 0a2a201fa2164e908f408bb07879b7cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:32.012119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:32.012981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-96974add-6ddb-4380-ba19-fd75a5eac867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:32.014598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1041/2081] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:31 2022] DELETE /identity/v3/projects/0a2a201fa2164e908f408bb07879b7cd => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:32.020440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e0790-de95-4202-8177-2f7797f60637 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:32.020959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e0790-de95-4202-8177-2f7797f60637 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:32.021058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e0790-de95-4202-8177-2f7797f60637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:32.042139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-524e0790-de95-4202-8177-2f7797f60637 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:32.043734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-524e0790-de95-4202-8177-2f7797f60637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:32.055508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:32.128385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1041/2082] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:32.137643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5090c-f15a-4e32-b9dd-78b6befb1ed0 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:32.137931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5090c-f15a-4e32-b9dd-78b6befb1ed0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:32.138296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5090c-f15a-4e32-b9dd-78b6befb1ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:32.160352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-a3f5090c-f15a-4e32-b9dd-78b6befb1ed0 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:32.161764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a3f5090c-f15a-4e32-b9dd-78b6befb1ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:32.169645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1042/2083] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:42:32.174071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e0790-de95-4202-8177-2f7797f60637 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:32.184064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e0790-de95-4202-8177-2f7797f60637 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:32.190735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73babf4-54f7-4e97-acd6-b79dafe9d6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:32.191510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73babf4-54f7-4e97-acd6-b79dafe9d6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:32.191800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73babf4-54f7-4e97-acd6-b79dafe9d6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:32.192082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73babf4-54f7-4e97-acd6-b79dafe9d6b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:32.192785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73babf4-54f7-4e97-acd6-b79dafe9d6b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:42:32.193097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73babf4-54f7-4e97-acd6-b79dafe9d6b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:32.194822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73babf4-54f7-4e97-acd6-b79dafe9d6b7 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:32.212050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1042/2084] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:42:32.212996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5090c-f15a-4e32-b9dd-78b6befb1ed0 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:32.223867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5090c-f15a-4e32-b9dd-78b6befb1ed0 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:32.228882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14a755-08c3-445f-acd4-fade158cd4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:32.229538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14a755-08c3-445f-acd4-fade158cd4e4 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:32.236195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14a755-08c3-445f-acd4-fade158cd4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:32.236195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14a755-08c3-445f-acd4-fade158cd4e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:32.236195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14a755-08c3-445f-acd4-fade158cd4e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:32.236195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14a755-08c3-445f-acd4-fade158cd4e4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:32.321860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1043/2085] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:32 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:32.326257 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73babf4-54f7-4e97-acd6-b79dafe9d6b7 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:32.336044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73babf4-54f7-4e97-acd6-b79dafe9d6b7 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:32.392971 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b28a23-c5ca-4cd4-b9fb-c66b93f347dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:32.393589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b28a23-c5ca-4cd4-b9fb-c66b93f347dc admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:32.393887 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b28a23-c5ca-4cd4-b9fb-c66b93f347dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:32.394182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b28a23-c5ca-4cd4-b9fb-c66b93f347dc admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:32.394850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b28a23-c5ca-4cd4-b9fb-c66b93f347dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:32.396774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b28a23-c5ca-4cd4-b9fb-c66b93f347dc admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:32.491958 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1043/2086] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:32 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:32.496400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14a755-08c3-445f-acd4-fade158cd4e4 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:32.508067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14a755-08c3-445f-acd4-fade158cd4e4 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:32.512910 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176112d-1e39-431e-b816-1c0ac2c533bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:32.513519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176112d-1e39-431e-b816-1c0ac2c533bb admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:32.513797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176112d-1e39-431e-b816-1c0ac2c533bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:32.514134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176112d-1e39-431e-b816-1c0ac2c533bb admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:32.514945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176112d-1e39-431e-b816-1c0ac2c533bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:32.516587 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176112d-1e39-431e-b816-1c0ac2c533bb admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:32.535729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1044/2087] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:32.538000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b28a23-c5ca-4cd4-b9fb-c66b93f347dc admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:32.549218 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b28a23-c5ca-4cd4-b9fb-c66b93f347dc admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:32.555892 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c22dc3-5275-461e-ab1f-f004aabcf9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:32.556611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c22dc3-5275-461e-ab1f-f004aabcf9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:32.556906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c22dc3-5275-461e-ab1f-f004aabcf9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:32.557228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c22dc3-5275-461e-ab1f-f004aabcf9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:32.557932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c22dc3-5275-461e-ab1f-f004aabcf9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:32.560239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c22dc3-5275-461e-ab1f-f004aabcf9a2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:32.574689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1044/2088] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:32.579153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176112d-1e39-431e-b816-1c0ac2c533bb admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:32.589546 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176112d-1e39-431e-b816-1c0ac2c533bb admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:32.595168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aeaa64-0fe4-40e3-8857-019bc53b6795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:32.595840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aeaa64-0fe4-40e3-8857-019bc53b6795 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:32.596368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aeaa64-0fe4-40e3-8857-019bc53b6795 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:32.597057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aeaa64-0fe4-40e3-8857-019bc53b6795 admin admin] PATH_INFO: `/v3/projects/0b4a4aaee2bd407e898a375748d94cd2/users/ea3046412b3f479b985d0714c2c56256/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:32.615441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aeaa64-0fe4-40e3-8857-019bc53b6795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4a4aaee2bd407e898a375748d94cd2, user_id=ea3046412b3f479b985d0714c2c56256, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:32.618978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aeaa64-0fe4-40e3-8857-019bc53b6795 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:32.704479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1045/2089] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:32 2022] PUT /identity/v3/projects/0b4a4aaee2bd407e898a375748d94cd2/users/ea3046412b3f479b985d0714c2c56256/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:32.716001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cdc66-05e2-424c-bf78-7c8569620720 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:32.716581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cdc66-05e2-424c-bf78-7c8569620720 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:32.716877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cdc66-05e2-424c-bf78-7c8569620720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:32.752359 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-dc1cdc66-05e2-424c-bf78-7c8569620720 None None] MFA Rules not processed for user `ea3046412b3f479b985d0714c2c56256`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:32.753130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-dc1cdc66-05e2-424c-bf78-7c8569620720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:32.841734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1045/2090] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:32.846077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aeaa64-0fe4-40e3-8857-019bc53b6795 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:32.918359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aeaa64-0fe4-40e3-8857-019bc53b6795 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:32.924088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319bdaf-7340-47a2-a083-59b6134956f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:32.924706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319bdaf-7340-47a2-a083-59b6134956f9 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:32.924991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319bdaf-7340-47a2-a083-59b6134956f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:32.925268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319bdaf-7340-47a2-a083-59b6134956f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:32.926304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9319bdaf-7340-47a2-a083-59b6134956f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:32.928234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9319bdaf-7340-47a2-a083-59b6134956f9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:33.005758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:33.017164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1046/2091] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:32 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:42:33.018363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cdc66-05e2-424c-bf78-7c8569620720 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:33.029215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cdc66-05e2-424c-bf78-7c8569620720 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:33.038470 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:33.090034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3afab-c3e0-40df-94a7-ef574f91b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:33.090673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3afab-c3e0-40df-94a7-ef574f91b598 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:33.090955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3afab-c3e0-40df-94a7-ef574f91b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:33.091231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3afab-c3e0-40df-94a7-ef574f91b598 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:33.091956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc3afab-c3e0-40df-94a7-ef574f91b598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:33.093897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc3afab-c3e0-40df-94a7-ef574f91b598 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:33.189538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1046/2092] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:33 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:33.193377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319bdaf-7340-47a2-a083-59b6134956f9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:33.209854 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319bdaf-7340-47a2-a083-59b6134956f9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:33.214692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27b6f9-abd9-4c5b-90b5-d5813bef75e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:33.215470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27b6f9-abd9-4c5b-90b5-d5813bef75e7 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:33.216472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27b6f9-abd9-4c5b-90b5-d5813bef75e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:33.216755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27b6f9-abd9-4c5b-90b5-d5813bef75e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:33.217479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27b6f9-abd9-4c5b-90b5-d5813bef75e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:33.219185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27b6f9-abd9-4c5b-90b5-d5813bef75e7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:33.231826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1047/2093] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:33.236176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3afab-c3e0-40df-94a7-ef574f91b598 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:33.247429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3afab-c3e0-40df-94a7-ef574f91b598 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:33.254653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68641ba-9830-4924-8df2-1311db7a1167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:33.255402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68641ba-9830-4924-8df2-1311db7a1167 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:33.255748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68641ba-9830-4924-8df2-1311db7a1167 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:33.256090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68641ba-9830-4924-8df2-1311db7a1167 admin admin] PATH_INFO: `/v3/projects/883c18e76c0f4ab29c8e39ff783760f3/users/c1d2df344a68499a939b3a8fab18fc65/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:33.284272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68641ba-9830-4924-8df2-1311db7a1167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=883c18e76c0f4ab29c8e39ff783760f3, user_id=c1d2df344a68499a939b3a8fab18fc65, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:33.287148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68641ba-9830-4924-8df2-1311db7a1167 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:33.372093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1047/2094] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:33 2022] PUT /identity/v3/projects/883c18e76c0f4ab29c8e39ff783760f3/users/c1d2df344a68499a939b3a8fab18fc65/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:33.378479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27b6f9-abd9-4c5b-90b5-d5813bef75e7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:33.388284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27b6f9-abd9-4c5b-90b5-d5813bef75e7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:33.447318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01f184-d0d6-407c-9fac-b511556d6024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:33.447943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01f184-d0d6-407c-9fac-b511556d6024 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:33.448218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01f184-d0d6-407c-9fac-b511556d6024 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:33.448492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01f184-d0d6-407c-9fac-b511556d6024 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:33.449185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01f184-d0d6-407c-9fac-b511556d6024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:33.450875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01f184-d0d6-407c-9fac-b511556d6024 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:33.461124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1048/2095] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:33.465465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68641ba-9830-4924-8df2-1311db7a1167 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:33.478550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68641ba-9830-4924-8df2-1311db7a1167 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:33.483420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b603a8-095e-456e-8ffd-4b3473a9c283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:33.484116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b603a8-095e-456e-8ffd-4b3473a9c283 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:33.484400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b603a8-095e-456e-8ffd-4b3473a9c283 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:33.484679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b603a8-095e-456e-8ffd-4b3473a9c283 admin admin] PATH_INFO: `/v3/projects/883c18e76c0f4ab29c8e39ff783760f3/users/c1d2df344a68499a939b3a8fab18fc65/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:33.487461 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b603a8-095e-456e-8ffd-4b3473a9c283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=883c18e76c0f4ab29c8e39ff783760f3, user_id=c1d2df344a68499a939b3a8fab18fc65, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:33.490534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b603a8-095e-456e-8ffd-4b3473a9c283 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:33.501589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1048/2096] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:33 2022] PUT /identity/v3/projects/883c18e76c0f4ab29c8e39ff783760f3/users/c1d2df344a68499a939b3a8fab18fc65/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:33.516561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937c2c2-d09e-44da-bd65-a0da5e3ac1f0 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:33.516881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937c2c2-d09e-44da-bd65-a0da5e3ac1f0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:33.517151 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937c2c2-d09e-44da-bd65-a0da5e3ac1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:33.547212 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c937c2c2-d09e-44da-bd65-a0da5e3ac1f0 None None] MFA Rules not processed for user `c1d2df344a68499a939b3a8fab18fc65`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:33.549215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c937c2c2-d09e-44da-bd65-a0da5e3ac1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:33.619560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1049/2097] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:33.630899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b603a8-095e-456e-8ffd-4b3473a9c283 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:33.632653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e7b603a8-095e-456e-8ffd-4b3473a9c283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:33.699050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b603a8-095e-456e-8ffd-4b3473a9c283 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:33.703261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354b7bf-6231-481a-a808-ea49f7eb81e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:33.704509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354b7bf-6231-481a-a808-ea49f7eb81e6 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:33.704866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354b7bf-6231-481a-a808-ea49f7eb81e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:33.705150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354b7bf-6231-481a-a808-ea49f7eb81e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:33.715721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354b7bf-6231-481a-a808-ea49f7eb81e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:33.717252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354b7bf-6231-481a-a808-ea49f7eb81e6 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:33.727509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1049/2098] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:33.873682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937c2c2-d09e-44da-bd65-a0da5e3ac1f0 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:33.937984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937c2c2-d09e-44da-bd65-a0da5e3ac1f0 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:33.942932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:33.943636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:33.943920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:33.944197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] PATH_INFO: `/v3/users/ea3046412b3f479b985d0714c2c56256` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:33.945373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea3046412b3f479b985d0714c2c56256)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:33.947404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:33.959060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:34.122321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea3046412b3f479b985d0714c2c56256'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:34.194030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea3046412b3f479b985d0714c2c56256'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:34.206795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea3046412b3f479b985d0714c2c56256'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:34.207517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea3046412b3f479b985d0714c2c56256'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:34.214684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea3046412b3f479b985d0714c2c56256'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:34.221982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1050/2099] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:33 2022] DELETE /identity/v3/users/ea3046412b3f479b985d0714c2c56256 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:34.226570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7610e7c-2e1f-4b36-b9fc-a8ec12879d34 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:34.227022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7610e7c-2e1f-4b36-b9fc-a8ec12879d34 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:34.227291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7610e7c-2e1f-4b36-b9fc-a8ec12879d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:34.246452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d7610e7c-2e1f-4b36-b9fc-a8ec12879d34 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:34.247671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d7610e7c-2e1f-4b36-b9fc-a8ec12879d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:34.309176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1050/2100] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:42:34.318898 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:34.320283 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:34.379626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565a68a-dba0-4ae7-819b-4819f7892916 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:34.383269 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3fdb5-dfb8-4784-9d02-6a97fa4e0f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:34.383891 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3fdb5-dfb8-4784-9d02-6a97fa4e0f2d service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:34.384195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3fdb5-dfb8-4784-9d02-6a97fa4e0f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:34.384473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3fdb5-dfb8-4784-9d02-6a97fa4e0f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:34.394426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3fdb5-dfb8-4784-9d02-6a97fa4e0f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:34.395981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3fdb5-dfb8-4784-9d02-6a97fa4e0f2d service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:34.405702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1051/2101] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:34.765012 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7610e7c-2e1f-4b36-b9fc-a8ec12879d34 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:34.767210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d7610e7c-2e1f-4b36-b9fc-a8ec12879d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:34.782776 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7610e7c-2e1f-4b36-b9fc-a8ec12879d34 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:34.788682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:34.789348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:34.789760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:34.790066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] PATH_INFO: `/v3/projects/0b4a4aaee2bd407e898a375748d94cd2` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:34.791869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b4a4aaee2bd407e898a375748d94cd2)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:34.801708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:34.803491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:34.909490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b4a4aaee2bd407e898a375748d94cd2', 'request_id': 'req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:34.915723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b4a4aaee2bd407e898a375748d94cd2', 'request_id': 'req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:35.007752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b4a4aaee2bd407e898a375748d94cd2', 'request_id': 'req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:35.015861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] The token cache is being invalidate because project 0b4a4aaee2bd407e898a375748d94cd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:35.016093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:35.016789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:35.018563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1051/2102] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:34 2022] DELETE /identity/v3/projects/0b4a4aaee2bd407e898a375748d94cd2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:35.022492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3fdb5-dfb8-4784-9d02-6a97fa4e0f2d service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:35.033046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3fdb5-dfb8-4784-9d02-6a97fa4e0f2d service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:35.089312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdafdc7-461d-4426-a684-d830bb41ab77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:35.090114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdafdc7-461d-4426-a684-d830bb41ab77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:35.090396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdafdc7-461d-4426-a684-d830bb41ab77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:35.090667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdafdc7-461d-4426-a684-d830bb41ab77 admin admin] PATH_INFO: `/v3/users/c1d2df344a68499a939b3a8fab18fc65` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:35.091840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdafdc7-461d-4426-a684-d830bb41ab77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d2df344a68499a939b3a8fab18fc65)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:35.093743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdafdc7-461d-4426-a684-d830bb41ab77 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:35.291485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:35.304725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2bdafdc7-461d-4426-a684-d830bb41ab77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d2df344a68499a939b3a8fab18fc65'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:35.380882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2bdafdc7-461d-4426-a684-d830bb41ab77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d2df344a68499a939b3a8fab18fc65'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:35.395330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2bdafdc7-461d-4426-a684-d830bb41ab77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d2df344a68499a939b3a8fab18fc65'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:35.396045 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2bdafdc7-461d-4426-a684-d830bb41ab77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d2df344a68499a939b3a8fab18fc65'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:35.403074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2bdafdc7-461d-4426-a684-d830bb41ab77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d2df344a68499a939b3a8fab18fc65'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:35.410056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1052/2103] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:35 2022] DELETE /identity/v3/users/c1d2df344a68499a939b3a8fab18fc65 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:42:35.899282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:35.901000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:35.930670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf5ae6-5be9-4a3b-be0e-83bcd9a0f332 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:35.987581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:35.988212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:35.988494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:35.988767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] PATH_INFO: `/v3/projects/883c18e76c0f4ab29c8e39ff783760f3` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:35.989966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=883c18e76c0f4ab29c8e39ff783760f3)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:35.991893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:36.120138 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '883c18e76c0f4ab29c8e39ff783760f3', 'request_id': 'req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:36.126981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '883c18e76c0f4ab29c8e39ff783760f3', 'request_id': 'req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:36.199895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '883c18e76c0f4ab29c8e39ff783760f3', 'request_id': 'req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:36.211306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] The token cache is being invalidate because project 883c18e76c0f4ab29c8e39ff783760f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:36.211539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:36.212196 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-227a18ff-f5ec-4f23-a0e3-3c64d431edcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:36.213723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1052/2104] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:35 2022] DELETE /identity/v3/projects/883c18e76c0f4ab29c8e39ff783760f3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:36.219257 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06593799-2088-43e5-aa3a-7f87f1d2b898 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:36.219740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06593799-2088-43e5-aa3a-7f87f1d2b898 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:36.219835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06593799-2088-43e5-aa3a-7f87f1d2b898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:36.241158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-06593799-2088-43e5-aa3a-7f87f1d2b898 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:36.242534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-06593799-2088-43e5-aa3a-7f87f1d2b898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:36.307716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1053/2105] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:42:36.316626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec926f-018e-464c-ba06-e12f525c43a0 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:36.317082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec926f-018e-464c-ba06-e12f525c43a0 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:36.317206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec926f-018e-464c-ba06-e12f525c43a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:36.343619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-dfec926f-018e-464c-ba06-e12f525c43a0 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:36.344838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-dfec926f-018e-464c-ba06-e12f525c43a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:36.353553 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1053/2106] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:36.357779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06593799-2088-43e5-aa3a-7f87f1d2b898 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:36.368678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06593799-2088-43e5-aa3a-7f87f1d2b898 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:36.373360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327b5cd-d0b8-4fb7-a02e-476df4a509e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:36.373934 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327b5cd-d0b8-4fb7-a02e-476df4a509e7 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:36.374244 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327b5cd-d0b8-4fb7-a02e-476df4a509e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:36.374524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327b5cd-d0b8-4fb7-a02e-476df4a509e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:36.375217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327b5cd-d0b8-4fb7-a02e-476df4a509e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:42:36.375525 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327b5cd-d0b8-4fb7-a02e-476df4a509e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:36.377110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327b5cd-d0b8-4fb7-a02e-476df4a509e7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:36.388685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1054/2107] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:42:36.393000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec926f-018e-464c-ba06-e12f525c43a0 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:36.405457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec926f-018e-464c-ba06-e12f525c43a0 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:36.411434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b6b49-5df5-4fe3-98f1-5b1faea3deae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:36.412229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b6b49-5df5-4fe3-98f1-5b1faea3deae admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:36.412513 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b6b49-5df5-4fe3-98f1-5b1faea3deae admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:36.412792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b6b49-5df5-4fe3-98f1-5b1faea3deae admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:36.413468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b6b49-5df5-4fe3-98f1-5b1faea3deae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:36.415486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b6b49-5df5-4fe3-98f1-5b1faea3deae admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:36.436110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:36.546135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1054/2108] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:36 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:36.551361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327b5cd-d0b8-4fb7-a02e-476df4a509e7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:36.562782 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327b5cd-d0b8-4fb7-a02e-476df4a509e7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:36.622632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbf4ca-5ab6-4893-ba27-eb506711c997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:36.623257 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbf4ca-5ab6-4893-ba27-eb506711c997 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:36.623609 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbf4ca-5ab6-4893-ba27-eb506711c997 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:36.623822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbf4ca-5ab6-4893-ba27-eb506711c997 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:36.624486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbf4ca-5ab6-4893-ba27-eb506711c997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:36.626657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbf4ca-5ab6-4893-ba27-eb506711c997 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:36.729922 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1055/2109] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:36 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:36.734149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b6b49-5df5-4fe3-98f1-5b1faea3deae admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:36.745585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b6b49-5df5-4fe3-98f1-5b1faea3deae admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:36.750447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4db36-ccbd-4ff7-8076-3e84ee180a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:36.750977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4db36-ccbd-4ff7-8076-3e84ee180a54 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:36.751258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4db36-ccbd-4ff7-8076-3e84ee180a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:36.751532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4db36-ccbd-4ff7-8076-3e84ee180a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:36.752222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4db36-ccbd-4ff7-8076-3e84ee180a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:36.753908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4db36-ccbd-4ff7-8076-3e84ee180a54 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:36.764336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1055/2110] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:36.773221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbf4ca-5ab6-4893-ba27-eb506711c997 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:36.783055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbf4ca-5ab6-4893-ba27-eb506711c997 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:36.789296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172f689-241b-4294-92c6-67658ef64ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:36.789624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172f689-241b-4294-92c6-67658ef64ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:36.790102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172f689-241b-4294-92c6-67658ef64ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:36.790457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172f689-241b-4294-92c6-67658ef64ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:36.791222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172f689-241b-4294-92c6-67658ef64ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:36.792870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172f689-241b-4294-92c6-67658ef64ae4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:36.804045 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1056/2111] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:36.807980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4db36-ccbd-4ff7-8076-3e84ee180a54 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:36.818423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4db36-ccbd-4ff7-8076-3e84ee180a54 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:36.823144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb460a-ef0a-4938-ac40-ae4a6eb8ad52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:36.823856 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb460a-ef0a-4938-ac40-ae4a6eb8ad52 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:36.824141 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb460a-ef0a-4938-ac40-ae4a6eb8ad52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:36.824421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb460a-ef0a-4938-ac40-ae4a6eb8ad52 admin admin] PATH_INFO: `/v3/projects/d4617ed7314840ae9dc2fc77afda8878/users/82aab70bc9af40bf9a1f7f584ffbba06/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:36.840727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb460a-ef0a-4938-ac40-ae4a6eb8ad52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4617ed7314840ae9dc2fc77afda8878, user_id=82aab70bc9af40bf9a1f7f584ffbba06, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:36.843471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb460a-ef0a-4938-ac40-ae4a6eb8ad52 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:36.915172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1056/2112] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:36 2022] PUT /identity/v3/projects/d4617ed7314840ae9dc2fc77afda8878/users/82aab70bc9af40bf9a1f7f584ffbba06/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:36.926498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837d958-9515-4b61-99ca-08e6387f0f98 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:36.926786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837d958-9515-4b61-99ca-08e6387f0f98 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:36.927061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837d958-9515-4b61-99ca-08e6387f0f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:36.954419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-3837d958-9515-4b61-99ca-08e6387f0f98 None None] MFA Rules not processed for user `82aab70bc9af40bf9a1f7f584ffbba06`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:36.955632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3837d958-9515-4b61-99ca-08e6387f0f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:36.960962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:37.025239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1057/2113] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:37.029781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb460a-ef0a-4938-ac40-ae4a6eb8ad52 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:37.092063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb460a-ef0a-4938-ac40-ae4a6eb8ad52 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:37.096824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b4620-6942-4753-821f-4940a6f415d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:37.098035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b4620-6942-4753-821f-4940a6f415d1 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:37.098350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b4620-6942-4753-821f-4940a6f415d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:37.098623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b4620-6942-4753-821f-4940a6f415d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:37.099287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b4620-6942-4753-821f-4940a6f415d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:37.101204 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b4620-6942-4753-821f-4940a6f415d1 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:37.176921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1057/2114] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:37 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:37.180833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837d958-9515-4b61-99ca-08e6387f0f98 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:37.191730 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837d958-9515-4b61-99ca-08e6387f0f98 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:37.248149 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0238606-3245-4938-aef5-91a73532d7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:37.248821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0238606-3245-4938-aef5-91a73532d7be admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:37.249036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0238606-3245-4938-aef5-91a73532d7be admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:37.249306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0238606-3245-4938-aef5-91a73532d7be admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:37.249992 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0238606-3245-4938-aef5-91a73532d7be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:37.252031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0238606-3245-4938-aef5-91a73532d7be admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:37.342584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1058/2115] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:37 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:42:37.346518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b4620-6942-4753-821f-4940a6f415d1 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:37.358020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b4620-6942-4753-821f-4940a6f415d1 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:37.363521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e2733-f704-47cc-884c-685b1abf5a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:37.364212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e2733-f704-47cc-884c-685b1abf5a5e admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:37.364593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e2733-f704-47cc-884c-685b1abf5a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:37.364947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e2733-f704-47cc-884c-685b1abf5a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:37.365713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e2733-f704-47cc-884c-685b1abf5a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:37.367362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e2733-f704-47cc-884c-685b1abf5a5e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:37.379156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1058/2116] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:37.383186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0238606-3245-4938-aef5-91a73532d7be admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:37.393613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0238606-3245-4938-aef5-91a73532d7be admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:37.398621 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60818e93-83a5-4f02-915b-fbd8976278f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:37.398916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60818e93-83a5-4f02-915b-fbd8976278f5 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:37.399193 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60818e93-83a5-4f02-915b-fbd8976278f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:37.399461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60818e93-83a5-4f02-915b-fbd8976278f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:37.400152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60818e93-83a5-4f02-915b-fbd8976278f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:37.401719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60818e93-83a5-4f02-915b-fbd8976278f5 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:37.412343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1059/2117] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:37.416681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e2733-f704-47cc-884c-685b1abf5a5e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:37.427692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e2733-f704-47cc-884c-685b1abf5a5e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:37.432412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424330b9-718a-471f-9d0b-273e7e9dad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:37.433072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424330b9-718a-471f-9d0b-273e7e9dad2d admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:37.433355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424330b9-718a-471f-9d0b-273e7e9dad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:37.433632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424330b9-718a-471f-9d0b-273e7e9dad2d admin admin] PATH_INFO: `/v3/projects/e8beb2c8870d42c6a700d4d369bea0d5/users/a672170d7e4d43e19c4b68647081ba66/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:37.448458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424330b9-718a-471f-9d0b-273e7e9dad2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8beb2c8870d42c6a700d4d369bea0d5, user_id=a672170d7e4d43e19c4b68647081ba66, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:37.451154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424330b9-718a-471f-9d0b-273e7e9dad2d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:37.554265 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1059/2118] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:37 2022] PUT /identity/v3/projects/e8beb2c8870d42c6a700d4d369bea0d5/users/a672170d7e4d43e19c4b68647081ba66/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:37.564672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3bf2-f5f5-4e36-a789-fc435aae72c0 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:37.564960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3bf2-f5f5-4e36-a789-fc435aae72c0 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:37.565238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3bf2-f5f5-4e36-a789-fc435aae72c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:37.592791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-b9ed3bf2-f5f5-4e36-a789-fc435aae72c0 None None] MFA Rules not processed for user `a672170d7e4d43e19c4b68647081ba66`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:37.593995 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b9ed3bf2-f5f5-4e36-a789-fc435aae72c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:37.656785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1060/2119] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:37.661993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424330b9-718a-471f-9d0b-273e7e9dad2d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:37.724394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424330b9-718a-471f-9d0b-273e7e9dad2d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:37.729285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dd0ab-43f3-4af6-a8a7-a839cf7b328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:37.730188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dd0ab-43f3-4af6-a8a7-a839cf7b328f admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:37.730475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dd0ab-43f3-4af6-a8a7-a839cf7b328f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:37.730754 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dd0ab-43f3-4af6-a8a7-a839cf7b328f admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:37.731500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4dd0ab-43f3-4af6-a8a7-a839cf7b328f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:37.733427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4dd0ab-43f3-4af6-a8a7-a839cf7b328f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:37.747035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:37.825856 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1060/2120] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:37 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:37.829662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed3bf2-f5f5-4e36-a789-fc435aae72c0 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:37.841042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed3bf2-f5f5-4e36-a789-fc435aae72c0 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:37.850989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:37.902655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23db53-8588-4252-9f50-77390c5f979c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:37.903302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23db53-8588-4252-9f50-77390c5f979c admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:37.903756 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23db53-8588-4252-9f50-77390c5f979c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:37.903869 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23db53-8588-4252-9f50-77390c5f979c admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:37.904588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23db53-8588-4252-9f50-77390c5f979c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:37.906579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23db53-8588-4252-9f50-77390c5f979c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:38.017982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1061/2121] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:37 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:42:38.022239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dd0ab-43f3-4af6-a8a7-a839cf7b328f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:38.033059 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dd0ab-43f3-4af6-a8a7-a839cf7b328f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:38.038644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03e5a2-557d-4de4-a614-c24e2f98acec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:38.039481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e5a2-557d-4de4-a614-c24e2f98acec admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:38.039974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e5a2-557d-4de4-a614-c24e2f98acec admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:38.040255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e5a2-557d-4de4-a614-c24e2f98acec admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:38.040961 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03e5a2-557d-4de4-a614-c24e2f98acec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:38.042603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03e5a2-557d-4de4-a614-c24e2f98acec admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:38.054811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1061/2122] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:38.058624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23db53-8588-4252-9f50-77390c5f979c admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:38.069367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23db53-8588-4252-9f50-77390c5f979c admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:38.074444 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d918fb2-5794-43d7-a7ac-546bff45380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:38.075141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d918fb2-5794-43d7-a7ac-546bff45380b admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:38.075444 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d918fb2-5794-43d7-a7ac-546bff45380b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:38.075771 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d918fb2-5794-43d7-a7ac-546bff45380b admin admin] PATH_INFO: `/v3/projects/8179e590fc1148de9016d2c84a964386/users/b5481ed0ca464c1ea92a40316e349d65/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:38.098852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d918fb2-5794-43d7-a7ac-546bff45380b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8179e590fc1148de9016d2c84a964386, user_id=b5481ed0ca464c1ea92a40316e349d65, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:38.101636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d918fb2-5794-43d7-a7ac-546bff45380b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:38.194484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1062/2123] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:38 2022] PUT /identity/v3/projects/8179e590fc1148de9016d2c84a964386/users/b5481ed0ca464c1ea92a40316e349d65/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:38.200675 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03e5a2-557d-4de4-a614-c24e2f98acec admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:38.210659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03e5a2-557d-4de4-a614-c24e2f98acec admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:38.267355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428abd80-eddc-4fc2-98fc-d22274dcd7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:38.267885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428abd80-eddc-4fc2-98fc-d22274dcd7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:38.268169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428abd80-eddc-4fc2-98fc-d22274dcd7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:38.268442 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428abd80-eddc-4fc2-98fc-d22274dcd7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:38.269642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428abd80-eddc-4fc2-98fc-d22274dcd7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:38.271471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428abd80-eddc-4fc2-98fc-d22274dcd7c1 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:38.281403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1062/2124] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:38.286441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d918fb2-5794-43d7-a7ac-546bff45380b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:38.296536 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d918fb2-5794-43d7-a7ac-546bff45380b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:38.301339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4578b9-5cae-4cd4-a4f2-42256342fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:38.301999 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4578b9-5cae-4cd4-a4f2-42256342fa62 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:38.302309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4578b9-5cae-4cd4-a4f2-42256342fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:38.302578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4578b9-5cae-4cd4-a4f2-42256342fa62 admin admin] PATH_INFO: `/v3/projects/8179e590fc1148de9016d2c84a964386/users/b5481ed0ca464c1ea92a40316e349d65/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:38.304997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4578b9-5cae-4cd4-a4f2-42256342fa62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8179e590fc1148de9016d2c84a964386, user_id=b5481ed0ca464c1ea92a40316e349d65, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:38.307706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4578b9-5cae-4cd4-a4f2-42256342fa62 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:38.318251 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1063/2125] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:38 2022] PUT /identity/v3/projects/8179e590fc1148de9016d2c84a964386/users/b5481ed0ca464c1ea92a40316e349d65/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:38.328688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c3692-915e-4b06-8516-edca0409f2d4 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:38.328976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c3692-915e-4b06-8516-edca0409f2d4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:38.329252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c3692-915e-4b06-8516-edca0409f2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:38.358465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a08c3692-915e-4b06-8516-edca0409f2d4 None None] MFA Rules not processed for user `b5481ed0ca464c1ea92a40316e349d65`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:38.359970 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a08c3692-915e-4b06-8516-edca0409f2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:38.430672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1063/2126] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:38.442240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4578b9-5cae-4cd4-a4f2-42256342fa62 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:38.443951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6a4578b9-5cae-4cd4-a4f2-42256342fa62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:38.504174 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4578b9-5cae-4cd4-a4f2-42256342fa62 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:38.507854 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251b4ea-ce2e-4cd7-af3d-d5d14feca6a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:38.508476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251b4ea-ce2e-4cd7-af3d-d5d14feca6a9 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:38.509123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251b4ea-ce2e-4cd7-af3d-d5d14feca6a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:38.509123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251b4ea-ce2e-4cd7-af3d-d5d14feca6a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:38.518243 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251b4ea-ce2e-4cd7-af3d-d5d14feca6a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:38.520180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251b4ea-ce2e-4cd7-af3d-d5d14feca6a9 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:38.530802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1064/2127] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:42:38.676379 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c3692-915e-4b06-8516-edca0409f2d4 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:38.686418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c3692-915e-4b06-8516-edca0409f2d4 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:38.690636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf1efb-49c4-40a5-9729-119229c790e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:38.691750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf1efb-49c4-40a5-9729-119229c790e3 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:38.692264 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf1efb-49c4-40a5-9729-119229c790e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:38.692653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf1efb-49c4-40a5-9729-119229c790e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:38.764772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf1efb-49c4-40a5-9729-119229c790e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:38.766719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf1efb-49c4-40a5-9729-119229c790e3 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:38.773528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:38.779471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1064/2128] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:43.142499 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251b4ea-ce2e-4cd7-af3d-d5d14feca6a9 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:43.258160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251b4ea-ce2e-4cd7-af3d-d5d14feca6a9 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:43.265007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:43.266508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:43.266744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:43.267020 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] PATH_INFO: `/v3/users/82aab70bc9af40bf9a1f7f584ffbba06` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:43.268228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82aab70bc9af40bf9a1f7f584ffbba06)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:43.270655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:43.282336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:43.530946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82aab70bc9af40bf9a1f7f584ffbba06'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:43.603307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82aab70bc9af40bf9a1f7f584ffbba06'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:43.615527 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82aab70bc9af40bf9a1f7f584ffbba06'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:43.616158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82aab70bc9af40bf9a1f7f584ffbba06'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:43.623177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82aab70bc9af40bf9a1f7f584ffbba06'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:43.629154 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1065/2129] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:43 2022] DELETE /identity/v3/users/82aab70bc9af40bf9a1f7f584ffbba06 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:43.633652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d211b4-c1aa-42fa-9514-fd17bbde4f34 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:43.634144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d211b4-c1aa-42fa-9514-fd17bbde4f34 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:43.634233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d211b4-c1aa-42fa-9514-fd17bbde4f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:43.653386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d8d211b4-c1aa-42fa-9514-fd17bbde4f34 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:43.654645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d8d211b4-c1aa-42fa-9514-fd17bbde4f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:43.717172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1065/2130] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:43.726531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:43.728118 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:43.794198 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed83258-07c7-40d3-bc01-5cbc3f829428 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:43.797785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1cea-4534-4da3-a763-70da8a1c29ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:43.798429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1cea-4534-4da3-a763-70da8a1c29ee service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:43.798711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1cea-4534-4da3-a763-70da8a1c29ee service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:43.798985 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1cea-4534-4da3-a763-70da8a1c29ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:43.808963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c1cea-4534-4da3-a763-70da8a1c29ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:43.810455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c1cea-4534-4da3-a763-70da8a1c29ee service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:43.821122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1066/2131] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:44.422621 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d211b4-c1aa-42fa-9514-fd17bbde4f34 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:44.424543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d8d211b4-c1aa-42fa-9514-fd17bbde4f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:44.451909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d211b4-c1aa-42fa-9514-fd17bbde4f34 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:44.467478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:44.468157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:44.468459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:44.468718 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] PATH_INFO: `/v3/projects/d4617ed7314840ae9dc2fc77afda8878` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:44.470448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4617ed7314840ae9dc2fc77afda8878)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:44.473723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:44.627102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4617ed7314840ae9dc2fc77afda8878', 'request_id': 'req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:44.635524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4617ed7314840ae9dc2fc77afda8878', 'request_id': 'req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:44.854888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4617ed7314840ae9dc2fc77afda8878', 'request_id': 'req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:44.866944 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] The token cache is being invalidate because project d4617ed7314840ae9dc2fc77afda8878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:44.867239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:44.867870 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:44.869653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1066/2132] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:44 2022] DELETE /identity/v3/projects/d4617ed7314840ae9dc2fc77afda8878 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:44.873714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1cea-4534-4da3-a763-70da8a1c29ee service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:44.883638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1cea-4534-4da3-a763-70da8a1c29ee service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:44.949484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c4189-e528-47bf-a4fd-b786d3fb6e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:44.950313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c4189-e528-47bf-a4fd-b786d3fb6e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:44.951788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c4189-e528-47bf-a4fd-b786d3fb6e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:44.951788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c4189-e528-47bf-a4fd-b786d3fb6e3b admin admin] PATH_INFO: `/v3/users/a672170d7e4d43e19c4b68647081ba66` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:44.952607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c4189-e528-47bf-a4fd-b786d3fb6e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a672170d7e4d43e19c4b68647081ba66)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:44.955093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c4189-e528-47bf-a4fd-b786d3fb6e3b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:45.224570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:45.237767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-5f3c4189-e528-47bf-a4fd-b786d3fb6e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a672170d7e4d43e19c4b68647081ba66'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:45.354272 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-5f3c4189-e528-47bf-a4fd-b786d3fb6e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a672170d7e4d43e19c4b68647081ba66'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:45.368941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-5f3c4189-e528-47bf-a4fd-b786d3fb6e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a672170d7e4d43e19c4b68647081ba66'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:45.369776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-5f3c4189-e528-47bf-a4fd-b786d3fb6e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a672170d7e4d43e19c4b68647081ba66'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:45.376939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-5f3c4189-e528-47bf-a4fd-b786d3fb6e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a672170d7e4d43e19c4b68647081ba66'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:45.384499 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1067/2133] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:44 2022] DELETE /identity/v3/users/a672170d7e4d43e19c4b68647081ba66 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:46.165800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:46.168157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:46.182333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd8f66-d5a1-499d-a7a2-8006171aaa8b admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:46.282224 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:46.282872 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:46.283242 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:46.283554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] PATH_INFO: `/v3/projects/e8beb2c8870d42c6a700d4d369bea0d5` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:46.284856 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8beb2c8870d42c6a700d4d369bea0d5)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:46.287573 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:46.393451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fbc4e-ff06-4c3a-9440-11b5caa7b715 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:46.393970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fbc4e-ff06-4c3a-9440-11b5caa7b715 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:46.394135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fbc4e-ff06-4c3a-9440-11b5caa7b715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:46.401284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:46.425109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-e42fbc4e-ff06-4c3a-9440-11b5caa7b715 None None] MFA Rules not processed for user `db4a45e9f40740c99cc49d9f2978332d`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:46.425334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8beb2c8870d42c6a700d4d369bea0d5', 'request_id': 'req-d57acf50-b41a-4f21-8446-0e0ee7b980a0'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:46.425442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e42fbc4e-ff06-4c3a-9440-11b5caa7b715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:46.428569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8beb2c8870d42c6a700d4d369bea0d5', 'request_id': 'req-d57acf50-b41a-4f21-8446-0e0ee7b980a0'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:46.488512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1068/2134] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:42:46.591918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8beb2c8870d42c6a700d4d369bea0d5', 'request_id': 'req-d57acf50-b41a-4f21-8446-0e0ee7b980a0'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:46.608381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] The token cache is being invalidate because project e8beb2c8870d42c6a700d4d369bea0d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:46.608381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:46.608381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:46.611193 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1067/2135] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:46 2022] DELETE /identity/v3/projects/e8beb2c8870d42c6a700d4d369bea0d5 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:46.612125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fbc4e-ff06-4c3a-9440-11b5caa7b715 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:46.613554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e42fbc4e-ff06-4c3a-9440-11b5caa7b715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:46.683254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fbc4e-ff06-4c3a-9440-11b5caa7b715 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:46.687876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c20c12-7a97-47c9-b7a7-b0c06536d6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:46.688902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c20c12-7a97-47c9-b7a7-b0c06536d6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:46.688902 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c20c12-7a97-47c9-b7a7-b0c06536d6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:46.689125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c20c12-7a97-47c9-b7a7-b0c06536d6e0 admin admin] PATH_INFO: `/v3/users/b5481ed0ca464c1ea92a40316e349d65` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:46.690308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c20c12-7a97-47c9-b7a7-b0c06536d6e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5481ed0ca464c1ea92a40316e349d65)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:46.692261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c20c12-7a97-47c9-b7a7-b0c06536d6e0 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:46.888374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a5c20c12-7a97-47c9-b7a7-b0c06536d6e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5481ed0ca464c1ea92a40316e349d65'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:46.999582 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a5c20c12-7a97-47c9-b7a7-b0c06536d6e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5481ed0ca464c1ea92a40316e349d65'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:47.002380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:47.014291 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a5c20c12-7a97-47c9-b7a7-b0c06536d6e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5481ed0ca464c1ea92a40316e349d65'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:47.015170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a5c20c12-7a97-47c9-b7a7-b0c06536d6e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5481ed0ca464c1ea92a40316e349d65'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:47.022108 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a5c20c12-7a97-47c9-b7a7-b0c06536d6e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5481ed0ca464c1ea92a40316e349d65'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:47.028176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1069/2136] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:46 2022] DELETE /identity/v3/users/b5481ed0ca464c1ea92a40316e349d65 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:42:47.497183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:47.499548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:47.513245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57acf50-b41a-4f21-8446-0e0ee7b980a0 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:47.594312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:47.594963 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:47.595250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:47.595520 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] PATH_INFO: `/v3/projects/8179e590fc1148de9016d2c84a964386` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:47.596833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8179e590fc1148de9016d2c84a964386)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:47.598899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:47.766602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8179e590fc1148de9016d2c84a964386', 'request_id': 'req-92b1c559-f263-4d76-b6f4-092c9c366128'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:47.773615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8179e590fc1148de9016d2c84a964386', 'request_id': 'req-92b1c559-f263-4d76-b6f4-092c9c366128'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:47.864476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8179e590fc1148de9016d2c84a964386', 'request_id': 'req-92b1c559-f263-4d76-b6f4-092c9c366128'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:47.875925 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] The token cache is being invalidate because project 8179e590fc1148de9016d2c84a964386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:47.876234 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:47.876901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-92b1c559-f263-4d76-b6f4-092c9c366128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:47.878476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1068/2137] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:47 2022] DELETE /identity/v3/projects/8179e590fc1148de9016d2c84a964386 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:47.885275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829d114-8ca9-4387-8e16-d61c6654b058 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:47.885766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829d114-8ca9-4387-8e16-d61c6654b058 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:47.885976 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829d114-8ca9-4387-8e16-d61c6654b058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:47.907422 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1829d114-8ca9-4387-8e16-d61c6654b058 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:47.908766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1829d114-8ca9-4387-8e16-d61c6654b058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:47.975551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1070/2138] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:42:47.984162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64410f19-dec0-4cc1-af19-aa73f9483c16 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:47.984450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64410f19-dec0-4cc1-af19-aa73f9483c16 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:47.984721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64410f19-dec0-4cc1-af19-aa73f9483c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:48.006212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-64410f19-dec0-4cc1-af19-aa73f9483c16 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:48.007338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-64410f19-dec0-4cc1-af19-aa73f9483c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:48.015585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1069/2139] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:48.020263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829d114-8ca9-4387-8e16-d61c6654b058 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:48.033182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829d114-8ca9-4387-8e16-d61c6654b058 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:48.038051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc678d4d-8d09-49db-acd2-39ea4a030963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:48.038550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc678d4d-8d09-49db-acd2-39ea4a030963 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:48.038805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc678d4d-8d09-49db-acd2-39ea4a030963 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:48.039081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc678d4d-8d09-49db-acd2-39ea4a030963 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:48.039822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc678d4d-8d09-49db-acd2-39ea4a030963 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:42:48.040079 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc678d4d-8d09-49db-acd2-39ea4a030963 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:48.041662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc678d4d-8d09-49db-acd2-39ea4a030963 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:48.057334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1071/2140] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:42:48.061508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64410f19-dec0-4cc1-af19-aa73f9483c16 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:48.073307 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64410f19-dec0-4cc1-af19-aa73f9483c16 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:48.078778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b45c5c-b28d-4b45-b21a-a4961c565539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:48.079719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b45c5c-b28d-4b45-b21a-a4961c565539 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:48.080015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b45c5c-b28d-4b45-b21a-a4961c565539 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:48.080532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b45c5c-b28d-4b45-b21a-a4961c565539 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:48.081705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b45c5c-b28d-4b45-b21a-a4961c565539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:48.084360 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b45c5c-b28d-4b45-b21a-a4961c565539 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:48.216276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1070/2141] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:48 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:48.218676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc678d4d-8d09-49db-acd2-39ea4a030963 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:48.231334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc678d4d-8d09-49db-acd2-39ea4a030963 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:48.321215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff98acd-fe1b-4376-ba72-af40a2c3ba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:48.321391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff98acd-fe1b-4376-ba72-af40a2c3ba99 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:48.321619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff98acd-fe1b-4376-ba72-af40a2c3ba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:48.321896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff98acd-fe1b-4376-ba72-af40a2c3ba99 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:48.322600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff98acd-fe1b-4376-ba72-af40a2c3ba99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:48.331122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff98acd-fe1b-4376-ba72-af40a2c3ba99 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:48.434954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1072/2142] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:48 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:48.439306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b45c5c-b28d-4b45-b21a-a4961c565539 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:48.450340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b45c5c-b28d-4b45-b21a-a4961c565539 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:48.455291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab04f8-dcb3-44af-b223-df4a42ee147d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:48.457734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab04f8-dcb3-44af-b223-df4a42ee147d admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:48.457734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab04f8-dcb3-44af-b223-df4a42ee147d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:48.457734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab04f8-dcb3-44af-b223-df4a42ee147d admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:48.457734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab04f8-dcb3-44af-b223-df4a42ee147d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:48.459100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab04f8-dcb3-44af-b223-df4a42ee147d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:48.469391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1071/2143] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:48.473575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff98acd-fe1b-4376-ba72-af40a2c3ba99 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:48.484572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff98acd-fe1b-4376-ba72-af40a2c3ba99 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:48.490152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88149acd-a375-4d8a-9204-54278c6c85c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:48.490912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149acd-a375-4d8a-9204-54278c6c85c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:48.491202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149acd-a375-4d8a-9204-54278c6c85c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:48.491482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149acd-a375-4d8a-9204-54278c6c85c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:48.492240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88149acd-a375-4d8a-9204-54278c6c85c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:48.494215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88149acd-a375-4d8a-9204-54278c6c85c0 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:48.505190 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1073/2144] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:48.509478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab04f8-dcb3-44af-b223-df4a42ee147d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:48.515937 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:48.520420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab04f8-dcb3-44af-b223-df4a42ee147d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:48.525840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c1d31-0ed4-4f57-ae61-679c3a1cc2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:48.526980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c1d31-0ed4-4f57-ae61-679c3a1cc2e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:48.527262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c1d31-0ed4-4f57-ae61-679c3a1cc2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:48.527539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c1d31-0ed4-4f57-ae61-679c3a1cc2e8 admin admin] PATH_INFO: `/v3/projects/3665a42b86de46deb14109d408d603d2/users/3b77b41ae9fe49b286c58c9f2358a9cc/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:48.543386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c1d31-0ed4-4f57-ae61-679c3a1cc2e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3665a42b86de46deb14109d408d603d2, user_id=3b77b41ae9fe49b286c58c9f2358a9cc, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:48.546153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c1d31-0ed4-4f57-ae61-679c3a1cc2e8 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:48.643559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1072/2145] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:48 2022] PUT /identity/v3/projects/3665a42b86de46deb14109d408d603d2/users/3b77b41ae9fe49b286c58c9f2358a9cc/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:48.655414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8a366-c953-43b4-99f3-df1c8a14307f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:48.655703 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8a366-c953-43b4-99f3-df1c8a14307f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:48.655977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8a366-c953-43b4-99f3-df1c8a14307f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:48.684015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-8ac8a366-c953-43b4-99f3-df1c8a14307f None None] MFA Rules not processed for user `3b77b41ae9fe49b286c58c9f2358a9cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:48.685213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8ac8a366-c953-43b4-99f3-df1c8a14307f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:48.726646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:48.769004 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1074/2146] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:48.774927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c1d31-0ed4-4f57-ae61-679c3a1cc2e8 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:48.847727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c1d31-0ed4-4f57-ae61-679c3a1cc2e8 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:48.853648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dd443-e1f1-4a96-a98f-38ed16e22135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:48.854633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dd443-e1f1-4a96-a98f-38ed16e22135 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:48.855014 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dd443-e1f1-4a96-a98f-38ed16e22135 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:48.855363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dd443-e1f1-4a96-a98f-38ed16e22135 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:48.856187 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dd443-e1f1-4a96-a98f-38ed16e22135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:48.858275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dd443-e1f1-4a96-a98f-38ed16e22135 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:49.005635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1073/2147] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:48 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:49.007109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8a366-c953-43b4-99f3-df1c8a14307f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:49.020360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8a366-c953-43b4-99f3-df1c8a14307f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:49.103716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c1811-559c-4966-a22a-07257af49d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:49.104344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c1811-559c-4966-a22a-07257af49d52 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:49.104686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c1811-559c-4966-a22a-07257af49d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:49.104896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c1811-559c-4966-a22a-07257af49d52 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:49.105578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c1811-559c-4966-a22a-07257af49d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:49.111932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c1811-559c-4966-a22a-07257af49d52 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:49.225874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1075/2148] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:49 2022] POST /identity/v3/users => generated 298 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:49.230058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dd443-e1f1-4a96-a98f-38ed16e22135 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:49.248481 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dd443-e1f1-4a96-a98f-38ed16e22135 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:49.257103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053af07-a917-4e7c-a332-723c4b5d9e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:49.258161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053af07-a917-4e7c-a332-723c4b5d9e2d admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:49.259291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053af07-a917-4e7c-a332-723c4b5d9e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:49.259969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053af07-a917-4e7c-a332-723c4b5d9e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:49.261081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053af07-a917-4e7c-a332-723c4b5d9e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:49.263864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053af07-a917-4e7c-a332-723c4b5d9e2d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:49.279210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1074/2149] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:49.284682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c1811-559c-4966-a22a-07257af49d52 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:49.307150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c1811-559c-4966-a22a-07257af49d52 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:49.310314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3668efe-fd5a-4273-975a-352f0ee625e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:49.311221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3668efe-fd5a-4273-975a-352f0ee625e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:49.311784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3668efe-fd5a-4273-975a-352f0ee625e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:49.312209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3668efe-fd5a-4273-975a-352f0ee625e9 admin admin] PATH_INFO: `/v3/projects/e6ea40b882ab460dbc8b45bd17dc9434/users/ff6223c6c41f45b8ad3ba3cd73e9df53/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:49.340525 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3668efe-fd5a-4273-975a-352f0ee625e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6ea40b882ab460dbc8b45bd17dc9434, user_id=ff6223c6c41f45b8ad3ba3cd73e9df53, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:49.344129 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3668efe-fd5a-4273-975a-352f0ee625e9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:49.439874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1076/2150] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:49 2022] PUT /identity/v3/projects/e6ea40b882ab460dbc8b45bd17dc9434/users/ff6223c6c41f45b8ad3ba3cd73e9df53/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:49.446598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053af07-a917-4e7c-a332-723c4b5d9e2d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:49.479944 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053af07-a917-4e7c-a332-723c4b5d9e2d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:49.603320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c4e26-3531-4026-8ce4-471347cd154f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:49.604878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c4e26-3531-4026-8ce4-471347cd154f admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:49.605369 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c4e26-3531-4026-8ce4-471347cd154f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:49.605837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c4e26-3531-4026-8ce4-471347cd154f admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:49.606905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c4e26-3531-4026-8ce4-471347cd154f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:49.608723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c4e26-3531-4026-8ce4-471347cd154f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:49.624110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1075/2151] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:49.628017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3668efe-fd5a-4273-975a-352f0ee625e9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:49.639911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3668efe-fd5a-4273-975a-352f0ee625e9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:49.648085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1b33c-c62e-4fff-b8a9-1ffa38d1c853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:49.648866 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1b33c-c62e-4fff-b8a9-1ffa38d1c853 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:49.649271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1b33c-c62e-4fff-b8a9-1ffa38d1c853 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:49.649688 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1b33c-c62e-4fff-b8a9-1ffa38d1c853 admin admin] PATH_INFO: `/v3/projects/e6ea40b882ab460dbc8b45bd17dc9434/users/ff6223c6c41f45b8ad3ba3cd73e9df53/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:49.654717 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1b33c-c62e-4fff-b8a9-1ffa38d1c853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6ea40b882ab460dbc8b45bd17dc9434, user_id=ff6223c6c41f45b8ad3ba3cd73e9df53, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:49.658373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1b33c-c62e-4fff-b8a9-1ffa38d1c853 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:49.683423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1077/2152] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:49 2022] PUT /identity/v3/projects/e6ea40b882ab460dbc8b45bd17dc9434/users/ff6223c6c41f45b8ad3ba3cd73e9df53/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:49.696602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b888d-5434-4612-afc7-9193c415e41e None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:49.697160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b888d-5434-4612-afc7-9193c415e41e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:49.697644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b888d-5434-4612-afc7-9193c415e41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:49.743007 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-b24b888d-5434-4612-afc7-9193c415e41e None None] MFA Rules not processed for user `ff6223c6c41f45b8ad3ba3cd73e9df53`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:49.745165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b24b888d-5434-4612-afc7-9193c415e41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:49.843201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:49.884961 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1076/2153] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:49.897599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1b33c-c62e-4fff-b8a9-1ffa38d1c853 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:49.900557 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-03d1b33c-c62e-4fff-b8a9-1ffa38d1c853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:50.018694 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1b33c-c62e-4fff-b8a9-1ffa38d1c853 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:50.024071 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e145d9-778c-4dd5-a633-0db8c0ad9ea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:50.024825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e145d9-778c-4dd5-a633-0db8c0ad9ea0 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:50.025217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e145d9-778c-4dd5-a633-0db8c0ad9ea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:50.025573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e145d9-778c-4dd5-a633-0db8c0ad9ea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:50.073696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:50.105577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e145d9-778c-4dd5-a633-0db8c0ad9ea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:50.107038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e145d9-778c-4dd5-a633-0db8c0ad9ea0 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:50.120836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1078/2154] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:52.203022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8153d3e-a3a0-4157-a8e6-723c50c3fc25 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:52.203555 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8153d3e-a3a0-4157-a8e6-723c50c3fc25 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:52.203555 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8153d3e-a3a0-4157-a8e6-723c50c3fc25 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:52.204932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1077/2155] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:42:52.253040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c39b9-e327-4f1d-bd76-16f77f730316 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:52.256087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c39b9-e327-4f1d-bd76-16f77f730316 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:52.257221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c39b9-e327-4f1d-bd76-16f77f730316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:52.313911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-247c39b9-e327-4f1d-bd76-16f77f730316 None None] MFA Rules not processed for user `fab36b186fa44ee18bf72228fc7ba758`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:52.315313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-247c39b9-e327-4f1d-bd76-16f77f730316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:52.423176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1079/2156] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:42:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:52.427401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8153d3e-a3a0-4157-a8e6-723c50c3fc25 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:52.429109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a8153d3e-a3a0-4157-a8e6-723c50c3fc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:52.447401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c39b9-e327-4f1d-bd76-16f77f730316 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:52.448972 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-247c39b9-e327-4f1d-bd76-16f77f730316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:52.464827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c39b9-e327-4f1d-bd76-16f77f730316 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:52.483716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68736ed3-8478-4742-af02-cbf9f69d6c48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:52.484355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68736ed3-8478-4742-af02-cbf9f69d6c48 service placement] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:52.484636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68736ed3-8478-4742-af02-cbf9f69d6c48 service placement] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:52.485081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68736ed3-8478-4742-af02-cbf9f69d6c48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:52.496111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68736ed3-8478-4742-af02-cbf9f69d6c48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:52.502851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68736ed3-8478-4742-af02-cbf9f69d6c48 service placement] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:52.524255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8153d3e-a3a0-4157-a8e6-723c50c3fc25 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:52.524854 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1080/2157] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:42:52.550638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78f634-f421-4c49-8a9a-6a5711189321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:52.551309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78f634-f421-4c49-8a9a-6a5711189321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:52.551809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78f634-f421-4c49-8a9a-6a5711189321 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:52.552084 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78f634-f421-4c49-8a9a-6a5711189321 admin admin] PATH_INFO: `/v3/users/d681a6cc3e5247c9bb2537a6cacb7dd4` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:52.556387 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78f634-f421-4c49-8a9a-6a5711189321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d681a6cc3e5247c9bb2537a6cacb7dd4)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:52.558710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78f634-f421-4c49-8a9a-6a5711189321 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:52.782027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5f78f634-f421-4c49-8a9a-6a5711189321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd681a6cc3e5247c9bb2537a6cacb7dd4'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:52.849437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5f78f634-f421-4c49-8a9a-6a5711189321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd681a6cc3e5247c9bb2537a6cacb7dd4'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:52.850118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5f78f634-f421-4c49-8a9a-6a5711189321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd681a6cc3e5247c9bb2537a6cacb7dd4'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:52.865552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5f78f634-f421-4c49-8a9a-6a5711189321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd681a6cc3e5247c9bb2537a6cacb7dd4'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:52.874179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5f78f634-f421-4c49-8a9a-6a5711189321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd681a6cc3e5247c9bb2537a6cacb7dd4'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:52.882679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1078/2158] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:42:52 2022] DELETE /identity/v3/users/d681a6cc3e5247c9bb2537a6cacb7dd4 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:53.165403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68736ed3-8478-4742-af02-cbf9f69d6c48 service placement] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:53.187242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68736ed3-8478-4742-af02-cbf9f69d6c48 service placement] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:53.254369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:53.255378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:53.255684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:53.255966 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] PATH_INFO: `/v3/projects/4c113838ba92441c89e4447d7d567b31` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:53.257146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c113838ba92441c89e4447d7d567b31)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:53.259125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:53.368047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c113838ba92441c89e4447d7d567b31', 'request_id': 'req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:53.496531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c113838ba92441c89e4447d7d567b31', 'request_id': 'req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:53.501193 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:53.503517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c113838ba92441c89e4447d7d567b31', 'request_id': 'req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:53.516338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] The token cache is being invalidate because project 4c113838ba92441c89e4447d7d567b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:42:53.516634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:53.517754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f7e26cff-7a88-4c09-8b44-e28a5d2ac92a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:42:53.519497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1081/2159] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:42:53 2022] DELETE /identity/v3/projects/4c113838ba92441c89e4447d7d567b31 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:53.525734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e501f2-80fa-478b-b1a8-c6b733f5002b None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:53.526616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e501f2-80fa-478b-b1a8-c6b733f5002b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:53.527071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e501f2-80fa-478b-b1a8-c6b733f5002b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:53.548902 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c9e501f2-80fa-478b-b1a8-c6b733f5002b None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:53.550616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c9e501f2-80fa-478b-b1a8-c6b733f5002b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:53.605527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:53.616961 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1079/2160] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:42:53.625102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4689b7-c455-42c9-8863-d285dae3defd None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:53.628929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4689b7-c455-42c9-8863-d285dae3defd None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:53.628929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4689b7-c455-42c9-8863-d285dae3defd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:53.646427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bb4689b7-c455-42c9-8863-d285dae3defd None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:53.647660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bb4689b7-c455-42c9-8863-d285dae3defd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:53.655690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1082/2161] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:53.660031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e501f2-80fa-478b-b1a8-c6b733f5002b None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:53.679297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e501f2-80fa-478b-b1a8-c6b733f5002b None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:53.679297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be118db3-738b-418c-aecb-873f213ec3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:53.679297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be118db3-738b-418c-aecb-873f213ec3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:53.679297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be118db3-738b-418c-aecb-873f213ec3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:53.679683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be118db3-738b-418c-aecb-873f213ec3c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:53.680116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be118db3-738b-418c-aecb-873f213ec3c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:42:53.680446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be118db3-738b-418c-aecb-873f213ec3c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:53.682036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be118db3-738b-418c-aecb-873f213ec3c7 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:53.694725 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1080/2162] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:42:53.699182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4689b7-c455-42c9-8863-d285dae3defd None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:53.709026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4689b7-c455-42c9-8863-d285dae3defd None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:53.713734 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6099b-eb49-4db6-91f9-e4bc4f8aac7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:53.714390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6099b-eb49-4db6-91f9-e4bc4f8aac7f admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:53.714670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6099b-eb49-4db6-91f9-e4bc4f8aac7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:53.714940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6099b-eb49-4db6-91f9-e4bc4f8aac7f admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:53.715686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6099b-eb49-4db6-91f9-e4bc4f8aac7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:53.718213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6099b-eb49-4db6-91f9-e4bc4f8aac7f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:53.803049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1083/2163] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:53 2022] POST /identity/v3/projects => generated 357 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:42:53.808864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be118db3-738b-418c-aecb-873f213ec3c7 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:53.820210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be118db3-738b-418c-aecb-873f213ec3c7 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:53.882024 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064bed7-7eab-4732-80ce-a7f811ee0215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:53.882659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064bed7-7eab-4732-80ce-a7f811ee0215 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:53.882938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064bed7-7eab-4732-80ce-a7f811ee0215 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:53.883209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064bed7-7eab-4732-80ce-a7f811ee0215 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:53.883881 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064bed7-7eab-4732-80ce-a7f811ee0215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:53.885800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064bed7-7eab-4732-80ce-a7f811ee0215 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:53.965907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1081/2164] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:53 2022] POST /identity/v3/users => generated 279 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:42:53.969823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6099b-eb49-4db6-91f9-e4bc4f8aac7f admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:53.980736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6099b-eb49-4db6-91f9-e4bc4f8aac7f admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:53.985655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f05cf-3c96-4fbf-b1c7-2faa2fde5654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:53.986297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f05cf-3c96-4fbf-b1c7-2faa2fde5654 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:53.986577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f05cf-3c96-4fbf-b1c7-2faa2fde5654 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:53.986850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f05cf-3c96-4fbf-b1c7-2faa2fde5654 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:53.988536 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f05cf-3c96-4fbf-b1c7-2faa2fde5654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:53.990394 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f05cf-3c96-4fbf-b1c7-2faa2fde5654 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:54.001972 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1084/2165] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:54.006227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064bed7-7eab-4732-80ce-a7f811ee0215 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:54.016807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064bed7-7eab-4732-80ce-a7f811ee0215 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:54.022181 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034bfb3-88f3-4db4-b76f-4a9749c587bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:54.023165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034bfb3-88f3-4db4-b76f-4a9749c587bd admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:54.023466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034bfb3-88f3-4db4-b76f-4a9749c587bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:54.023738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034bfb3-88f3-4db4-b76f-4a9749c587bd admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:54.024433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034bfb3-88f3-4db4-b76f-4a9749c587bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:54.026054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034bfb3-88f3-4db4-b76f-4a9749c587bd admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:54.039318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1082/2166] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:54.043318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f05cf-3c96-4fbf-b1c7-2faa2fde5654 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:54.054044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f05cf-3c96-4fbf-b1c7-2faa2fde5654 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:54.058872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77333af8-1735-4465-a3d4-8732dd9c3818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:54.059662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77333af8-1735-4465-a3d4-8732dd9c3818 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:54.060008 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77333af8-1735-4465-a3d4-8732dd9c3818 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:54.060301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77333af8-1735-4465-a3d4-8732dd9c3818 admin admin] PATH_INFO: `/v3/projects/4806bebe2c89464fa72b43a1c91f9a8b/users/8307225a68db47a7b84e43a9b93f6f51/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:54.075723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77333af8-1735-4465-a3d4-8732dd9c3818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4806bebe2c89464fa72b43a1c91f9a8b, user_id=8307225a68db47a7b84e43a9b93f6f51, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:54.078640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77333af8-1735-4465-a3d4-8732dd9c3818 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:54.144833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1085/2167] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:54 2022] PUT /identity/v3/projects/4806bebe2c89464fa72b43a1c91f9a8b/users/8307225a68db47a7b84e43a9b93f6f51/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:54.157651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7dd62-216b-43ee-b1ad-2b5ba23e8d3f None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:54.158086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7dd62-216b-43ee-b1ad-2b5ba23e8d3f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:54.158372 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7dd62-216b-43ee-b1ad-2b5ba23e8d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:54.187273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-14b7dd62-216b-43ee-b1ad-2b5ba23e8d3f None None] MFA Rules not processed for user `8307225a68db47a7b84e43a9b93f6f51`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:54.188498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-14b7dd62-216b-43ee-b1ad-2b5ba23e8d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:54.254137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1083/2168] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:54.258623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77333af8-1735-4465-a3d4-8732dd9c3818 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:54.324748 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77333af8-1735-4465-a3d4-8732dd9c3818 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:54.329606 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103f612-32e2-45f4-b38f-994289ce15e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:54.330252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f612-32e2-45f4-b38f-994289ce15e6 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:54.330541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f612-32e2-45f4-b38f-994289ce15e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:54.330821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f612-32e2-45f4-b38f-994289ce15e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:54.331501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103f612-32e2-45f4-b38f-994289ce15e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:54.334360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103f612-32e2-45f4-b38f-994289ce15e6 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:54.521789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1086/2169] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:42:54 2022] POST /identity/v3/projects => generated 357 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:42:54.526095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7dd62-216b-43ee-b1ad-2b5ba23e8d3f None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:54.538723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7dd62-216b-43ee-b1ad-2b5ba23e8d3f None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:54.598268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:54.609838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6a270-df76-4c0c-b216-39c742cc5f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:54.610563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a270-df76-4c0c-b216-39c742cc5f1b admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:54.610908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a270-df76-4c0c-b216-39c742cc5f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:54.611248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a270-df76-4c0c-b216-39c742cc5f1b admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:54.611935 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6a270-df76-4c0c-b216-39c742cc5f1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:54.613931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6a270-df76-4c0c-b216-39c742cc5f1b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:54.695528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1084/2170] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:42:54 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:42:54.702478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103f612-32e2-45f4-b38f-994289ce15e6 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:54.713660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103f612-32e2-45f4-b38f-994289ce15e6 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:54.718354 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd99ee-da86-43d8-aead-8912678702e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:54.718964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd99ee-da86-43d8-aead-8912678702e2 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:54.719273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd99ee-da86-43d8-aead-8912678702e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:54.719547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd99ee-da86-43d8-aead-8912678702e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:54.720240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd99ee-da86-43d8-aead-8912678702e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:54.721851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd99ee-da86-43d8-aead-8912678702e2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:54.734539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1087/2171] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:54.738900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6a270-df76-4c0c-b216-39c742cc5f1b admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:54.749915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6a270-df76-4c0c-b216-39c742cc5f1b admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:54.754553 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8937a-5228-4ebb-90aa-48387a9d6776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:54.755284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8937a-5228-4ebb-90aa-48387a9d6776 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:54.755569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8937a-5228-4ebb-90aa-48387a9d6776 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:54.755842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8937a-5228-4ebb-90aa-48387a9d6776 admin admin] PATH_INFO: `/v3/projects/aa930c5eab444ae7aee813d6acc9b0bd/users/5315912babe4420eb05d70c363dd7224/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:54.771413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8937a-5228-4ebb-90aa-48387a9d6776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa930c5eab444ae7aee813d6acc9b0bd, user_id=5315912babe4420eb05d70c363dd7224, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:54.774072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8937a-5228-4ebb-90aa-48387a9d6776 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:54.834296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1085/2172] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:54 2022] PUT /identity/v3/projects/aa930c5eab444ae7aee813d6acc9b0bd/users/5315912babe4420eb05d70c363dd7224/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:54.838438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd99ee-da86-43d8-aead-8912678702e2 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:54.848390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd99ee-da86-43d8-aead-8912678702e2 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:55.047577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dc811-ede2-41e9-ba5a-3d177767a996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:55.048189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dc811-ede2-41e9-ba5a-3d177767a996 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:55.048468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dc811-ede2-41e9-ba5a-3d177767a996 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:55.048738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dc811-ede2-41e9-ba5a-3d177767a996 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:55.049486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dc811-ede2-41e9-ba5a-3d177767a996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:55.051087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dc811-ede2-41e9-ba5a-3d177767a996 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:55.052479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:42:55.061357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1088/2173] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:42:55.065603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8937a-5228-4ebb-90aa-48387a9d6776 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:55.076352 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8937a-5228-4ebb-90aa-48387a9d6776 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:55.096005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc1ad0-c2ea-487b-a5fa-6a38e6d98498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:55.096005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1ad0-c2ea-487b-a5fa-6a38e6d98498 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:55.096005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1ad0-c2ea-487b-a5fa-6a38e6d98498 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:55.096005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1ad0-c2ea-487b-a5fa-6a38e6d98498 admin admin] PATH_INFO: `/v3/projects/aa930c5eab444ae7aee813d6acc9b0bd/users/5315912babe4420eb05d70c363dd7224/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:55.096945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc1ad0-c2ea-487b-a5fa-6a38e6d98498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa930c5eab444ae7aee813d6acc9b0bd, user_id=5315912babe4420eb05d70c363dd7224, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:55.096945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc1ad0-c2ea-487b-a5fa-6a38e6d98498 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:55.100782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1086/2174] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:42:55 2022] PUT /identity/v3/projects/aa930c5eab444ae7aee813d6acc9b0bd/users/5315912babe4420eb05d70c363dd7224/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:42:55.111282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9cf73-974f-4f89-85cf-ebe67545469a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:55.111572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9cf73-974f-4f89-85cf-ebe67545469a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:55.111850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9cf73-974f-4f89-85cf-ebe67545469a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:55.139731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-5db9cf73-974f-4f89-85cf-ebe67545469a None None] MFA Rules not processed for user `5315912babe4420eb05d70c363dd7224`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:42:55.140924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5db9cf73-974f-4f89-85cf-ebe67545469a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:55.205904 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1089/2175] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:42:55 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:42:55.218929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc1ad0-c2ea-487b-a5fa-6a38e6d98498 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:55.220733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9cbc1ad0-c2ea-487b-a5fa-6a38e6d98498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:55.281976 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc1ad0-c2ea-487b-a5fa-6a38e6d98498 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:55.285728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bbb8a-0866-4c3d-b95d-8ddacc1f1947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:55.286529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bbb8a-0866-4c3d-b95d-8ddacc1f1947 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:55.286888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bbb8a-0866-4c3d-b95d-8ddacc1f1947 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:55.287140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bbb8a-0866-4c3d-b95d-8ddacc1f1947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:55.297450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bbb8a-0866-4c3d-b95d-8ddacc1f1947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:55.299593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bbb8a-0866-4c3d-b95d-8ddacc1f1947 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:55.311175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1087/2176] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:55 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:55.542206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9cf73-974f-4f89-85cf-ebe67545469a None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:55.552353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9cf73-974f-4f89-85cf-ebe67545469a None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:55.556337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad74bf3-e9ac-4a3b-a866-155acc5f8133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:55.556944 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74bf3-e9ac-4a3b-a866-155acc5f8133 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:55.557277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74bf3-e9ac-4a3b-a866-155acc5f8133 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:55.557535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74bf3-e9ac-4a3b-a866-155acc5f8133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:55.617763 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad74bf3-e9ac-4a3b-a866-155acc5f8133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:55.619486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad74bf3-e9ac-4a3b-a866-155acc5f8133 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:55.628582 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1090/2177] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:55 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:42:58.857378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bbb8a-0866-4c3d-b95d-8ddacc1f1947 service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:42:58.858848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e25bbb8a-0866-4c3d-b95d-8ddacc1f1947 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:58.870867 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bbb8a-0866-4c3d-b95d-8ddacc1f1947 service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:42:58.874646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307b808-2fab-44b2-ad1f-70852bde203d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:42:58.875260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b808-2fab-44b2-ad1f-70852bde203d service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:42:58.875549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b808-2fab-44b2-ad1f-70852bde203d service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:42:58.875923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b808-2fab-44b2-ad1f-70852bde203d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:42:58.877826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5307b808-2fab-44b2-ad1f-70852bde203d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:42:58.943591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307b808-2fab-44b2-ad1f-70852bde203d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:42:58.945105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307b808-2fab-44b2-ad1f-70852bde203d service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:42:58.955047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1088/2178] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:42:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:43:05.624728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=64 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:06.078253 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad74bf3-e9ac-4a3b-a866-155acc5f8133 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:06.079648 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7ad74bf3-e9ac-4a3b-a866-155acc5f8133 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:06.148783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad74bf3-e9ac-4a3b-a866-155acc5f8133 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:06.153376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:06.154049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:06.154355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:06.158352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] PATH_INFO: `/v3/users/3b77b41ae9fe49b286c58c9f2358a9cc` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:06.158352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b77b41ae9fe49b286c58c9f2358a9cc)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:06.158352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:06.361946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b77b41ae9fe49b286c58c9f2358a9cc'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:06.484847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b77b41ae9fe49b286c58c9f2358a9cc'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:06.498360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b77b41ae9fe49b286c58c9f2358a9cc'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:06.499007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b77b41ae9fe49b286c58c9f2358a9cc'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:06.506319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b77b41ae9fe49b286c58c9f2358a9cc'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:06.512402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1091/2179] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:43:06 2022] DELETE /identity/v3/users/3b77b41ae9fe49b286c58c9f2358a9cc => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:06.516951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525fb81-540e-4a11-9c1f-4c128fc7305a None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:06.517625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525fb81-540e-4a11-9c1f-4c128fc7305a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:06.517827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525fb81-540e-4a11-9c1f-4c128fc7305a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:06.538355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-e525fb81-540e-4a11-9c1f-4c128fc7305a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:06.539783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e525fb81-540e-4a11-9c1f-4c128fc7305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:06.572774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:06.597916 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1089/2180] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:06.607904 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:06.609407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:06.688213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239c4a4-764b-4177-b32f-155109a2f147 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:06.692203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81ac39-3720-4f11-9349-910b6f8c0493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:06.693633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81ac39-3720-4f11-9349-910b6f8c0493 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:06.693979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81ac39-3720-4f11-9349-910b6f8c0493 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:06.694339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81ac39-3720-4f11-9349-910b6f8c0493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:06.704792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81ac39-3720-4f11-9349-910b6f8c0493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:06.706361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81ac39-3720-4f11-9349-910b6f8c0493 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:06.717738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1092/2181] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:43:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:43:07.243114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525fb81-540e-4a11-9c1f-4c128fc7305a None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:07.244848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e525fb81-540e-4a11-9c1f-4c128fc7305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:07.259928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525fb81-540e-4a11-9c1f-4c128fc7305a None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:07.265325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:07.266078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:07.266368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:07.269353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] PATH_INFO: `/v3/projects/3665a42b86de46deb14109d408d603d2` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:07.269353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3665a42b86de46deb14109d408d603d2)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:07.269743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:07.382251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3665a42b86de46deb14109d408d603d2', 'request_id': 'req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:07.388528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3665a42b86de46deb14109d408d603d2', 'request_id': 'req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:07.450558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3665a42b86de46deb14109d408d603d2', 'request_id': 'req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:07.462320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] The token cache is being invalidate because project 3665a42b86de46deb14109d408d603d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:43:07.462612 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:07.463253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:07.464780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1090/2182] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:43:07 2022] DELETE /identity/v3/projects/3665a42b86de46deb14109d408d603d2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:07.468735 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81ac39-3720-4f11-9349-910b6f8c0493 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:07.479022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81ac39-3720-4f11-9349-910b6f8c0493 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:07.497077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:07.537336 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20c536-2ec5-4850-9946-0f886e636994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:07.538568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20c536-2ec5-4850-9946-0f886e636994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:07.538568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20c536-2ec5-4850-9946-0f886e636994 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:07.544034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20c536-2ec5-4850-9946-0f886e636994 admin admin] PATH_INFO: `/v3/users/ff6223c6c41f45b8ad3ba3cd73e9df53` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:07.544034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20c536-2ec5-4850-9946-0f886e636994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6223c6c41f45b8ad3ba3cd73e9df53)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:07.544034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20c536-2ec5-4850-9946-0f886e636994 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:07.736723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9e20c536-2ec5-4850-9946-0f886e636994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6223c6c41f45b8ad3ba3cd73e9df53'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:07.791921 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9e20c536-2ec5-4850-9946-0f886e636994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6223c6c41f45b8ad3ba3cd73e9df53'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:07.804218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9e20c536-2ec5-4850-9946-0f886e636994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6223c6c41f45b8ad3ba3cd73e9df53'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:07.804881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9e20c536-2ec5-4850-9946-0f886e636994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6223c6c41f45b8ad3ba3cd73e9df53'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:07.811581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9e20c536-2ec5-4850-9946-0f886e636994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6223c6c41f45b8ad3ba3cd73e9df53'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:07.817918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1093/2183] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:43:07 2022] DELETE /identity/v3/users/ff6223c6c41f45b8ad3ba3cd73e9df53 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:43:08.304203 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:08.305680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:08.319473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef2b4b-14ba-40ee-8e11-b8068e979fb6 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:08.385767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:08.385767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:08.386215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:08.386215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] PATH_INFO: `/v3/projects/e6ea40b882ab460dbc8b45bd17dc9434` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:08.387399 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6ea40b882ab460dbc8b45bd17dc9434)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:08.389315 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:08.502335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6ea40b882ab460dbc8b45bd17dc9434', 'request_id': 'req-6132d60e-3830-45ae-908a-483c1a361a0b'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:08.509117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6ea40b882ab460dbc8b45bd17dc9434', 'request_id': 'req-6132d60e-3830-45ae-908a-483c1a361a0b'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:08.564458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6ea40b882ab460dbc8b45bd17dc9434', 'request_id': 'req-6132d60e-3830-45ae-908a-483c1a361a0b'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:08.569219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:08.576332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] The token cache is being invalidate because project e6ea40b882ab460dbc8b45bd17dc9434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:43:08.576615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:08.577266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6132d60e-3830-45ae-908a-483c1a361a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:08.578837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1091/2184] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:43:08 2022] DELETE /identity/v3/projects/e6ea40b882ab460dbc8b45bd17dc9434 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:08.584975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958708d-af5c-4fb9-b35f-c46b366bd581 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:08.585272 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958708d-af5c-4fb9-b35f-c46b366bd581 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:08.585581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958708d-af5c-4fb9-b35f-c46b366bd581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:08.606460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-5958708d-af5c-4fb9-b35f-c46b366bd581 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:08.607715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5958708d-af5c-4fb9-b35f-c46b366bd581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:08.668555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1094/2185] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:43:08.676982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427b7a6-eccf-4782-9fd4-e76d95d50470 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:08.677359 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427b7a6-eccf-4782-9fd4-e76d95d50470 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:08.677638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427b7a6-eccf-4782-9fd4-e76d95d50470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:08.697536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-3427b7a6-eccf-4782-9fd4-e76d95d50470 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:08.698771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3427b7a6-eccf-4782-9fd4-e76d95d50470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:08.707307 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1092/2186] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:08.712749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958708d-af5c-4fb9-b35f-c46b366bd581 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:08.724095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958708d-af5c-4fb9-b35f-c46b366bd581 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:08.728786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7a73d-8b00-4215-90cf-b3ff3427c003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:08.729395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7a73d-8b00-4215-90cf-b3ff3427c003 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:08.729675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7a73d-8b00-4215-90cf-b3ff3427c003 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:08.729978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7a73d-8b00-4215-90cf-b3ff3427c003 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:08.730676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7a73d-8b00-4215-90cf-b3ff3427c003 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:43:08.730978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7a73d-8b00-4215-90cf-b3ff3427c003 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:08.732565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7a73d-8b00-4215-90cf-b3ff3427c003 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:08.745242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1095/2187] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:43:08.749799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427b7a6-eccf-4782-9fd4-e76d95d50470 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:08.761330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427b7a6-eccf-4782-9fd4-e76d95d50470 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:08.764432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95b44d-8a66-4d45-9a2f-f18f4b065f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:08.765047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b44d-8a66-4d45-9a2f-f18f4b065f8f admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:08.765332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b44d-8a66-4d45-9a2f-f18f4b065f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:08.765607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b44d-8a66-4d45-9a2f-f18f4b065f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:08.766313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95b44d-8a66-4d45-9a2f-f18f4b065f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:08.768225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95b44d-8a66-4d45-9a2f-f18f4b065f8f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:08.866080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1093/2188] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:43:08 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:43:08.870194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7a73d-8b00-4215-90cf-b3ff3427c003 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:08.882141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7a73d-8b00-4215-90cf-b3ff3427c003 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:08.912034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:08.943083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c49cf-26f1-449a-bda2-bb0be0160152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:08.943723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c49cf-26f1-449a-bda2-bb0be0160152 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:08.944002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c49cf-26f1-449a-bda2-bb0be0160152 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:08.944300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c49cf-26f1-449a-bda2-bb0be0160152 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:08.944956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c49cf-26f1-449a-bda2-bb0be0160152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:08.946871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c49cf-26f1-449a-bda2-bb0be0160152 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:09.068124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1096/2189] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:43:08 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:43:09.072508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95b44d-8a66-4d45-9a2f-f18f4b065f8f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:09.084521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95b44d-8a66-4d45-9a2f-f18f4b065f8f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:09.089545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2aebf-6e0c-455f-ae00-f621f91854b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:09.090335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2aebf-6e0c-455f-ae00-f621f91854b8 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:09.090678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2aebf-6e0c-455f-ae00-f621f91854b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:09.091160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2aebf-6e0c-455f-ae00-f621f91854b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:09.091928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2aebf-6e0c-455f-ae00-f621f91854b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:09.093728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2aebf-6e0c-455f-ae00-f621f91854b8 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:09.105687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1094/2190] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:43:09.109678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c49cf-26f1-449a-bda2-bb0be0160152 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:09.119887 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c49cf-26f1-449a-bda2-bb0be0160152 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:09.124715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f1aa1-a2c3-4c0e-a6f7-09356e89ec3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:09.125318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f1aa1-a2c3-4c0e-a6f7-09356e89ec3b admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:09.125598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f1aa1-a2c3-4c0e-a6f7-09356e89ec3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:09.125872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f1aa1-a2c3-4c0e-a6f7-09356e89ec3b admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:09.128815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f1aa1-a2c3-4c0e-a6f7-09356e89ec3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:09.128815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f1aa1-a2c3-4c0e-a6f7-09356e89ec3b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:09.141623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1097/2191] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:43:09.142469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2aebf-6e0c-455f-ae00-f621f91854b8 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:09.153549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2aebf-6e0c-455f-ae00-f621f91854b8 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:09.165523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038aee99-2c61-4395-9fe7-c303c7c94f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:09.166630 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038aee99-2c61-4395-9fe7-c303c7c94f7d admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:09.167120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038aee99-2c61-4395-9fe7-c303c7c94f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:09.167597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038aee99-2c61-4395-9fe7-c303c7c94f7d admin admin] PATH_INFO: `/v3/projects/7a43db10eda0469382e7ac3de7aea5f9/users/212a87162222437d941c666d5fb7c23e/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:09.194936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038aee99-2c61-4395-9fe7-c303c7c94f7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a43db10eda0469382e7ac3de7aea5f9, user_id=212a87162222437d941c666d5fb7c23e, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:09.197678 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038aee99-2c61-4395-9fe7-c303c7c94f7d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:09.263325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1095/2192] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:43:09 2022] PUT /identity/v3/projects/7a43db10eda0469382e7ac3de7aea5f9/users/212a87162222437d941c666d5fb7c23e/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:09.274852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb80803-f3f5-41f0-a5b3-55bab085d511 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:09.275066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb80803-f3f5-41f0-a5b3-55bab085d511 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:09.275400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb80803-f3f5-41f0-a5b3-55bab085d511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:09.304849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-aeb80803-f3f5-41f0-a5b3-55bab085d511 None None] MFA Rules not processed for user `212a87162222437d941c666d5fb7c23e`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:09.306136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-aeb80803-f3f5-41f0-a5b3-55bab085d511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:09.374752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1098/2193] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:09.387655 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038aee99-2c61-4395-9fe7-c303c7c94f7d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:09.389414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-038aee99-2c61-4395-9fe7-c303c7c94f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:09.463416 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038aee99-2c61-4395-9fe7-c303c7c94f7d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:09.468237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c88070-1559-4f24-9b0a-776e255d5536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:09.469135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c88070-1559-4f24-9b0a-776e255d5536 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:09.469422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c88070-1559-4f24-9b0a-776e255d5536 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:09.469700 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c88070-1559-4f24-9b0a-776e255d5536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:09.480362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c88070-1559-4f24-9b0a-776e255d5536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:09.482238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c88070-1559-4f24-9b0a-776e255d5536 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:09.493429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1096/2194] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:43:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:43:09.517339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb80803-f3f5-41f0-a5b3-55bab085d511 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:09.590250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb80803-f3f5-41f0-a5b3-55bab085d511 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:09.596429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:09.597283 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:09.597634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:09.597984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] PATH_INFO: `/v3/users/212a87162222437d941c666d5fb7c23e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:09.599555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=212a87162222437d941c666d5fb7c23e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:09.601448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:09.796820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:09.807687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212a87162222437d941c666d5fb7c23e'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:09.884746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212a87162222437d941c666d5fb7c23e'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:09.897636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212a87162222437d941c666d5fb7c23e'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:09.898374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212a87162222437d941c666d5fb7c23e'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:09.905161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212a87162222437d941c666d5fb7c23e'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:09.912634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1099/2195] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:43:09 2022] DELETE /identity/v3/users/212a87162222437d941c666d5fb7c23e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:09.917291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4d5ac-4773-4f12-9f96-6d4091f18394 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:09.917588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4d5ac-4773-4f12-9f96-6d4091f18394 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:09.917872 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4d5ac-4773-4f12-9f96-6d4091f18394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:09.939941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-2cd4d5ac-4773-4f12-9f96-6d4091f18394 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:09.941199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2cd4d5ac-4773-4f12-9f96-6d4091f18394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:10.007009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1097/2196] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:10.017524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:10.018975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:10.088896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9373c-967b-4a96-8e6b-c36b869e75a7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:10.093819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fe1a5-351e-44e8-81f2-29ae3ad415fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:10.094473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fe1a5-351e-44e8-81f2-29ae3ad415fe service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:10.094758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fe1a5-351e-44e8-81f2-29ae3ad415fe service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:10.095023 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fe1a5-351e-44e8-81f2-29ae3ad415fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:10.106706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fe1a5-351e-44e8-81f2-29ae3ad415fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:10.108126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fe1a5-351e-44e8-81f2-29ae3ad415fe service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:10.119866 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1100/2197] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:43:10.462878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189010e4-fd4b-4dfc-b04a-14c0d93ee3e1 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:10.462878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189010e4-fd4b-4dfc-b04a-14c0d93ee3e1 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:10.462878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189010e4-fd4b-4dfc-b04a-14c0d93ee3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:10.488938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-189010e4-fd4b-4dfc-b04a-14c0d93ee3e1 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:10.490732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-189010e4-fd4b-4dfc-b04a-14c0d93ee3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:10.543326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:10.577430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1098/2198] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:43:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:10.638158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fe1a5-351e-44e8-81f2-29ae3ad415fe service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:10.640169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f34fe1a5-351e-44e8-81f2-29ae3ad415fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:10.740819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189010e4-fd4b-4dfc-b04a-14c0d93ee3e1 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:10.742448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-189010e4-fd4b-4dfc-b04a-14c0d93ee3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:10.758334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fe1a5-351e-44e8-81f2-29ae3ad415fe service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:10.761815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189010e4-fd4b-4dfc-b04a-14c0d93ee3e1 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:10.765178 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1d68c-5578-40ed-8025-b3fdeb889dfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:10.765810 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1d68c-5578-40ed-8025-b3fdeb889dfe service swift] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:10.766217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1d68c-5578-40ed-8025-b3fdeb889dfe service swift] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:10.766396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1d68c-5578-40ed-8025-b3fdeb889dfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:10.767196 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:10.768088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:10.768522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:10.768868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] PATH_INFO: `/v3/projects/7a43db10eda0469382e7ac3de7aea5f9` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:10.770992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a43db10eda0469382e7ac3de7aea5f9)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:10.772421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:10.777973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1d68c-5578-40ed-8025-b3fdeb889dfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:10.779619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1d68c-5578-40ed-8025-b3fdeb889dfe service swift] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:10.789893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1101/2199] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:43:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:43:10.908965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a43db10eda0469382e7ac3de7aea5f9', 'request_id': 'req-ac3fecba-182a-48fe-8208-d6a45be18e73'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:10.915523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a43db10eda0469382e7ac3de7aea5f9', 'request_id': 'req-ac3fecba-182a-48fe-8208-d6a45be18e73'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:10.988145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a43db10eda0469382e7ac3de7aea5f9', 'request_id': 'req-ac3fecba-182a-48fe-8208-d6a45be18e73'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:11.003752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] The token cache is being invalidate because project 7a43db10eda0469382e7ac3de7aea5f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:43:11.004047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:11.005270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ac3fecba-182a-48fe-8208-d6a45be18e73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:11.006897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1099/2200] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:43:10 2022] DELETE /identity/v3/projects/7a43db10eda0469382e7ac3de7aea5f9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:43:11.012053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985fc73-e33b-4577-b1d4-c843221cda0f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:11.012562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985fc73-e33b-4577-b1d4-c843221cda0f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:11.012654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985fc73-e33b-4577-b1d4-c843221cda0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:11.034899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-2985fc73-e33b-4577-b1d4-c843221cda0f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:11.041175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2985fc73-e33b-4577-b1d4-c843221cda0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:11.113380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:11.138981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1102/2201] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:11.213359 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17b717-21d7-4be3-b675-0c0dd753575e None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:11.213705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17b717-21d7-4be3-b675-0c0dd753575e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:11.213924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17b717-21d7-4be3-b675-0c0dd753575e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:11.235436 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-eb17b717-21d7-4be3-b675-0c0dd753575e None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:11.236794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-eb17b717-21d7-4be3-b675-0c0dd753575e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:11.246079 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1100/2202] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:11.250510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985fc73-e33b-4577-b1d4-c843221cda0f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:11.261141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985fc73-e33b-4577-b1d4-c843221cda0f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:11.266282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad2977-bec8-4709-a6ec-3f9cbf9f5aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:11.266894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad2977-bec8-4709-a6ec-3f9cbf9f5aef admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:11.267235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad2977-bec8-4709-a6ec-3f9cbf9f5aef admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:11.267510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad2977-bec8-4709-a6ec-3f9cbf9f5aef admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:11.268198 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad2977-bec8-4709-a6ec-3f9cbf9f5aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:43:11.268500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad2977-bec8-4709-a6ec-3f9cbf9f5aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:11.270791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad2977-bec8-4709-a6ec-3f9cbf9f5aef admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:11.285668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1103/2203] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:43:11.288402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17b717-21d7-4be3-b675-0c0dd753575e None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:11.298057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17b717-21d7-4be3-b675-0c0dd753575e None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:11.306401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c54b6-d8cb-4bc6-ba63-adfdf3c8b1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:11.307044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c54b6-d8cb-4bc6-ba63-adfdf3c8b1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:11.307742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c54b6-d8cb-4bc6-ba63-adfdf3c8b1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:11.308022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c54b6-d8cb-4bc6-ba63-adfdf3c8b1e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:11.308693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c54b6-d8cb-4bc6-ba63-adfdf3c8b1e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:11.310704 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c54b6-d8cb-4bc6-ba63-adfdf3c8b1e0 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:11.430234 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1101/2204] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:43:11 2022] POST /identity/v3/projects => generated 361 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:43:11.434559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad2977-bec8-4709-a6ec-3f9cbf9f5aef admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:11.445867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad2977-bec8-4709-a6ec-3f9cbf9f5aef admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:11.509188 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3de3a-f128-46a6-b356-27083d6b2804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:11.509808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3de3a-f128-46a6-b356-27083d6b2804 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:11.510129 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3de3a-f128-46a6-b356-27083d6b2804 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:11.510417 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3de3a-f128-46a6-b356-27083d6b2804 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:11.511052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3de3a-f128-46a6-b356-27083d6b2804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:11.512984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3de3a-f128-46a6-b356-27083d6b2804 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:11.639396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1104/2205] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:43:11 2022] POST /identity/v3/users => generated 281 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:43:11.643521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c54b6-d8cb-4bc6-ba63-adfdf3c8b1e0 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:11.655170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c54b6-d8cb-4bc6-ba63-adfdf3c8b1e0 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:11.660474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1b83e-1c2c-4d8d-9198-32ada6d8a8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:11.661263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1b83e-1c2c-4d8d-9198-32ada6d8a8dc admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:11.661781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1b83e-1c2c-4d8d-9198-32ada6d8a8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:11.662114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1b83e-1c2c-4d8d-9198-32ada6d8a8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:11.662831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1b83e-1c2c-4d8d-9198-32ada6d8a8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:11.664435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1b83e-1c2c-4d8d-9198-32ada6d8a8dc admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:11.675081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1102/2206] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:43:11.679745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3de3a-f128-46a6-b356-27083d6b2804 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:11.688382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2c661-b1f1-428c-8d7e-8fcebe6c6abd None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:11.688699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2c661-b1f1-428c-8d7e-8fcebe6c6abd None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:11.688974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2c661-b1f1-428c-8d7e-8fcebe6c6abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:11.690024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3de3a-f128-46a6-b356-27083d6b2804 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:11.695545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bca3a-afd8-41e4-818b-8b153b0232a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:11.696163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bca3a-afd8-41e4-818b-8b153b0232a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:11.696448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bca3a-afd8-41e4-818b-8b153b0232a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:11.696721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bca3a-afd8-41e4-818b-8b153b0232a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:11.697423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bca3a-afd8-41e4-818b-8b153b0232a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:11.699082 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bca3a-afd8-41e4-818b-8b153b0232a7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:11.710538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1105/2207] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:43:11.710846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1fb2c661-b1f1-428c-8d7e-8fcebe6c6abd None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:11.712120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1fb2c661-b1f1-428c-8d7e-8fcebe6c6abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:11.716446 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bca3a-afd8-41e4-818b-8b153b0232a7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:11.727263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bca3a-afd8-41e4-818b-8b153b0232a7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:11.732397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c92a29-d50c-4611-a770-7247a3648f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:11.733069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c92a29-d50c-4611-a770-7247a3648f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:11.733350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c92a29-d50c-4611-a770-7247a3648f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:11.733725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c92a29-d50c-4611-a770-7247a3648f6b admin admin] PATH_INFO: `/v3/projects/b9213316edac4b3aae018c0b6c88170c/users/b7222604393f4ffab48e0e49c845d800/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:11.752805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c92a29-d50c-4611-a770-7247a3648f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9213316edac4b3aae018c0b6c88170c, user_id=b7222604393f4ffab48e0e49c845d800, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:11.755741 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c92a29-d50c-4611-a770-7247a3648f6b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:11.781305 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1103/2208] 10.222.0.73 () {58 vars in 967 bytes} [Thu Aug 4 21:43:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:43:11.794460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2c661-b1f1-428c-8d7e-8fcebe6c6abd None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:11.795968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1fb2c661-b1f1-428c-8d7e-8fcebe6c6abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:11.830898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:11.843568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1106/2209] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:43:11 2022] PUT /identity/v3/projects/b9213316edac4b3aae018c0b6c88170c/users/b7222604393f4ffab48e0e49c845d800/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:43:11.858683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366abe1-53f3-4da5-90a5-c4745c8b2132 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:11.860175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366abe1-53f3-4da5-90a5-c4745c8b2132 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:11.861184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366abe1-53f3-4da5-90a5-c4745c8b2132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:11.880919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2c661-b1f1-428c-8d7e-8fcebe6c6abd None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:11.898647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-0366abe1-53f3-4da5-90a5-c4745c8b2132 None None] MFA Rules not processed for user `b7222604393f4ffab48e0e49c845d800`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:11.899865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-0366abe1-53f3-4da5-90a5-c4745c8b2132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:11.946410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa93553-6919-4e11-b4a0-5e05a380a1f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:11.947232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa93553-6919-4e11-b4a0-5e05a380a1f5 service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:11.948018 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa93553-6919-4e11-b4a0-5e05a380a1f5 service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:11.948341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa93553-6919-4e11-b4a0-5e05a380a1f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:11.976081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1107/2210] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:43:12.010447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366abe1-53f3-4da5-90a5-c4745c8b2132 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:12.011858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-0366abe1-53f3-4da5-90a5-c4745c8b2132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:12.018067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa93553-6919-4e11-b4a0-5e05a380a1f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:12.018067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa93553-6919-4e11-b4a0-5e05a380a1f5 service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:12.026843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1104/2211] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:43:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:43:12.035002 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:12.080996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366abe1-53f3-4da5-90a5-c4745c8b2132 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:12.085443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26eb60-2c0b-4e17-9bfb-c77297840140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:12.086127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26eb60-2c0b-4e17-9bfb-c77297840140 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:12.086413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26eb60-2c0b-4e17-9bfb-c77297840140 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:12.086685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26eb60-2c0b-4e17-9bfb-c77297840140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:12.095873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26eb60-2c0b-4e17-9bfb-c77297840140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:12.097345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26eb60-2c0b-4e17-9bfb-c77297840140 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:12.108050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1108/2212] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:43:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:43:12.124739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa93553-6919-4e11-b4a0-5e05a380a1f5 service swift] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:12.196392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa93553-6919-4e11-b4a0-5e05a380a1f5 service swift] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:12.203631 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:12.204594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:12.205040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:12.205469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] PATH_INFO: `/v3/users/b7222604393f4ffab48e0e49c845d800` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:12.207294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7222604393f4ffab48e0e49c845d800)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:12.211193 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:12.225924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:12.381129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7222604393f4ffab48e0e49c845d800'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:12.455064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7222604393f4ffab48e0e49c845d800'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:12.456195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7222604393f4ffab48e0e49c845d800'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:12.469403 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7222604393f4ffab48e0e49c845d800'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:12.476982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7222604393f4ffab48e0e49c845d800'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:12.484252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1105/2213] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:43:12 2022] DELETE /identity/v3/users/b7222604393f4ffab48e0e49c845d800 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:12.488824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede50589-67ea-4a99-b21c-7683523a03b9 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:12.489193 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede50589-67ea-4a99-b21c-7683523a03b9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:12.490050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede50589-67ea-4a99-b21c-7683523a03b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:12.510599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ede50589-67ea-4a99-b21c-7683523a03b9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:12.511926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ede50589-67ea-4a99-b21c-7683523a03b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:12.573548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1109/2214] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:12.588309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:12.590017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:12.653898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b9936-358d-4440-bd87-67f51fa34989 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:12.659130 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b542-ed73-49d8-86a6-ed151f62d255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:12.659758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b542-ed73-49d8-86a6-ed151f62d255 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:12.660035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b542-ed73-49d8-86a6-ed151f62d255 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:12.660293 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b542-ed73-49d8-86a6-ed151f62d255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:12.669958 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b542-ed73-49d8-86a6-ed151f62d255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:12.671448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b542-ed73-49d8-86a6-ed151f62d255 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:12.681489 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1106/2215] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:43:12.706573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400983d5-2388-4317-9888-c03449e1a1c4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:12.706888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400983d5-2388-4317-9888-c03449e1a1c4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:12.707170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400983d5-2388-4317-9888-c03449e1a1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:12.730568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-400983d5-2388-4317-9888-c03449e1a1c4 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:12.732439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-400983d5-2388-4317-9888-c03449e1a1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:12.791600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1110/2216] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:43:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:12.803558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b542-ed73-49d8-86a6-ed151f62d255 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:12.805011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6e95b542-ed73-49d8-86a6-ed151f62d255 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:12.892358 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b542-ed73-49d8-86a6-ed151f62d255 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:12.899499 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fc9e5-9177-48a9-8cb0-ae52b6b84bed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:12.900405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc9e5-9177-48a9-8cb0-ae52b6b84bed service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:12.900721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc9e5-9177-48a9-8cb0-ae52b6b84bed service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:12.901118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc9e5-9177-48a9-8cb0-ae52b6b84bed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:12.912815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fc9e5-9177-48a9-8cb0-ae52b6b84bed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:12.914843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fc9e5-9177-48a9-8cb0-ae52b6b84bed service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:12.924710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1107/2217] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:43:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:43:13.138481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400983d5-2388-4317-9888-c03449e1a1c4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:13.140410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-400983d5-2388-4317-9888-c03449e1a1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:13.157063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400983d5-2388-4317-9888-c03449e1a1c4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:13.164297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:13.165001 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:13.165286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:13.165690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] PATH_INFO: `/v3/projects/b9213316edac4b3aae018c0b6c88170c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:13.166927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9213316edac4b3aae018c0b6c88170c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:13.169185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:13.338363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9213316edac4b3aae018c0b6c88170c', 'request_id': 'req-7d45d557-f41b-4c58-98ec-63be6fd35571'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:13.460767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9213316edac4b3aae018c0b6c88170c', 'request_id': 'req-7d45d557-f41b-4c58-98ec-63be6fd35571'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:13.467579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:13.471073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9213316edac4b3aae018c0b6c88170c', 'request_id': 'req-7d45d557-f41b-4c58-98ec-63be6fd35571'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:13.485775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] The token cache is being invalidate because project b9213316edac4b3aae018c0b6c88170c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:43:13.486173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:13.486835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:13.490193 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1111/2218] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:43:13 2022] DELETE /identity/v3/projects/b9213316edac4b3aae018c0b6c88170c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:13.689674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7715a9-7c4e-42a3-9a41-6b744e53d92c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:13.690305 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7715a9-7c4e-42a3-9a41-6b744e53d92c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:13.697388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7715a9-7c4e-42a3-9a41-6b744e53d92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:13.716416 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-9b7715a9-7c4e-42a3-9a41-6b744e53d92c None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:13.717919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9b7715a9-7c4e-42a3-9a41-6b744e53d92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:13.747417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:13.789294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1108/2219] 10.222.0.73 () {58 vars in 967 bytes} [Thu Aug 4 21:43:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:13.802509 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:13.803904 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:13.874782 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45d557-f41b-4c58-98ec-63be6fd35571 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:13.880137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8f48f-2be5-493a-a7a6-f6209594c431 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:13.880767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8f48f-2be5-493a-a7a6-f6209594c431 service swift] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:13.881229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8f48f-2be5-493a-a7a6-f6209594c431 service swift] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:13.881966 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8f48f-2be5-493a-a7a6-f6209594c431 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:13.890262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8f48f-2be5-493a-a7a6-f6209594c431 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:13.891940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8f48f-2be5-493a-a7a6-f6209594c431 service swift] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:13.901531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1112/2220] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:43:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:43:18.135103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309567d-db1b-4022-8a57-6a6577b6de3d None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:18.135103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309567d-db1b-4022-8a57-6a6577b6de3d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:18.135103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309567d-db1b-4022-8a57-6a6577b6de3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:18.153177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d309567d-db1b-4022-8a57-6a6577b6de3d None None] MFA Rules not processed for user `d9bc42bec393430b9b3e9399a8c4f95e`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:18.154473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d309567d-db1b-4022-8a57-6a6577b6de3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:18.209706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1109/2221] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:21.704111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c72497-7084-466d-b703-0183deea31b2 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:21.704667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c72497-7084-466d-b703-0183deea31b2 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:21.704667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c72497-7084-466d-b703-0183deea31b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:21.724375 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-59c72497-7084-466d-b703-0183deea31b2 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:21.725569 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-59c72497-7084-466d-b703-0183deea31b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:21.732865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1113/2222] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:43:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:21.747545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309567d-db1b-4022-8a57-6a6577b6de3d None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:21.762047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309567d-db1b-4022-8a57-6a6577b6de3d None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:21.767919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614ffa0-7523-46bf-9808-405802ae37f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:21.768544 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614ffa0-7523-46bf-9808-405802ae37f5 service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:21.768911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614ffa0-7523-46bf-9808-405802ae37f5 service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:21.769783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614ffa0-7523-46bf-9808-405802ae37f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:21.778736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614ffa0-7523-46bf-9808-405802ae37f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:21.780405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614ffa0-7523-46bf-9808-405802ae37f5 service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:21.790743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1110/2223] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:43:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:43:22.120405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c72497-7084-466d-b703-0183deea31b2 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:22.121907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-59c72497-7084-466d-b703-0183deea31b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:22.209326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c72497-7084-466d-b703-0183deea31b2 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:22.215495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:22.216297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:22.216720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:22.217040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] PATH_INFO: `/v3/users/7487daf672c04b1bb052a8a5e4875d61` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:22.218689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7487daf672c04b1bb052a8a5e4875d61)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:22.220858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:22.461376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7487daf672c04b1bb052a8a5e4875d61'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:22.592551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7487daf672c04b1bb052a8a5e4875d61'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:22.610572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7487daf672c04b1bb052a8a5e4875d61'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:22.611346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7487daf672c04b1bb052a8a5e4875d61'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:22.617727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:22.621247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7487daf672c04b1bb052a8a5e4875d61'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:22.628245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1114/2224] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:43:22 2022] DELETE /identity/v3/users/7487daf672c04b1bb052a8a5e4875d61 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:22.633389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9109d9-c1cd-4d71-8deb-f2fa91091c06 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:22.634424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9109d9-c1cd-4d71-8deb-f2fa91091c06 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:22.634830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9109d9-c1cd-4d71-8deb-f2fa91091c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:22.667995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-4d9109d9-c1cd-4d71-8deb-f2fa91091c06 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:22.669307 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4d9109d9-c1cd-4d71-8deb-f2fa91091c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:22.746995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1111/2225] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:22.760718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:22.762827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:22.853376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3768dce-eedd-4151-baba-d4c95b9c67d4 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:22.859480 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b2498-df20-41fb-8fa5-6fdef5ac57bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:22.860779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b2498-df20-41fb-8fa5-6fdef5ac57bf service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:22.861084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b2498-df20-41fb-8fa5-6fdef5ac57bf service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:22.861374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b2498-df20-41fb-8fa5-6fdef5ac57bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:22.876893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b2498-df20-41fb-8fa5-6fdef5ac57bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:22.881402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b2498-df20-41fb-8fa5-6fdef5ac57bf service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:22.899412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1115/2226] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:43:23.344603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9109d9-c1cd-4d71-8deb-f2fa91091c06 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:23.347043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4d9109d9-c1cd-4d71-8deb-f2fa91091c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:23.366388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9109d9-c1cd-4d71-8deb-f2fa91091c06 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:23.382226 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:23.383741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:23.384312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:23.384677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] PATH_INFO: `/v3/projects/f738ce353ad649709334f71983434d3c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:23.387137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f738ce353ad649709334f71983434d3c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:23.391807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:23.565192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f738ce353ad649709334f71983434d3c', 'request_id': 'req-652d6466-5ec6-48b3-ae48-6971c511ff78'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:23.586125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f738ce353ad649709334f71983434d3c', 'request_id': 'req-652d6466-5ec6-48b3-ae48-6971c511ff78'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:23.589483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:23.660235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b2498-df20-41fb-8fa5-6fdef5ac57bf service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:23.663423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bf0b2498-df20-41fb-8fa5-6fdef5ac57bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:23.684942 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f738ce353ad649709334f71983434d3c', 'request_id': 'req-652d6466-5ec6-48b3-ae48-6971c511ff78'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:23.699973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] The token cache is being invalidate because project f738ce353ad649709334f71983434d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:43:23.699973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:23.699973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-652d6466-5ec6-48b3-ae48-6971c511ff78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:23.703382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1112/2227] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:43:23 2022] DELETE /identity/v3/projects/f738ce353ad649709334f71983434d3c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:23.736719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b2498-df20-41fb-8fa5-6fdef5ac57bf service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:23.895453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:23.897685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:23.898175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:23.906400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] PATH_INFO: `/v3/users/8307225a68db47a7b84e43a9b93f6f51` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:23.910236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8307225a68db47a7b84e43a9b93f6f51)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:23.912294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:24.141895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8307225a68db47a7b84e43a9b93f6f51'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:24.227774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8307225a68db47a7b84e43a9b93f6f51'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:24.245092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8307225a68db47a7b84e43a9b93f6f51'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:24.245908 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8307225a68db47a7b84e43a9b93f6f51'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:24.254144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8307225a68db47a7b84e43a9b93f6f51'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:24.264646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1116/2228] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:43:23 2022] DELETE /identity/v3/users/8307225a68db47a7b84e43a9b93f6f51 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:24.275441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b839-81bb-44dd-9b12-439840958a97 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:24.275890 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b839-81bb-44dd-9b12-439840958a97 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:24.276397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b839-81bb-44dd-9b12-439840958a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:24.304012 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a123b839-81bb-44dd-9b12-439840958a97 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:24.305331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a123b839-81bb-44dd-9b12-439840958a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:24.400684 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1113/2229] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:24.413585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:24.415312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:24.456308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:24.520281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf0f63-e0cd-49ce-ad3a-01e88d31e9c4 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:24.526578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cace9b-483d-4653-80d2-7491be7f3a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:24.527781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cace9b-483d-4653-80d2-7491be7f3a35 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:24.528133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cace9b-483d-4653-80d2-7491be7f3a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:24.528511 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cace9b-483d-4653-80d2-7491be7f3a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:24.546695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cace9b-483d-4653-80d2-7491be7f3a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:24.548163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cace9b-483d-4653-80d2-7491be7f3a35 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:24.565729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1117/2230] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:43:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:43:25.076225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123b839-81bb-44dd-9b12-439840958a97 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:25.077919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a123b839-81bb-44dd-9b12-439840958a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:25.094671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123b839-81bb-44dd-9b12-439840958a97 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:25.100242 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:25.101028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:25.101418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:25.101796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] PATH_INFO: `/v3/projects/4806bebe2c89464fa72b43a1c91f9a8b` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:25.103148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4806bebe2c89464fa72b43a1c91f9a8b)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:25.105200 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:25.206175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4806bebe2c89464fa72b43a1c91f9a8b', 'request_id': 'req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:25.213524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4806bebe2c89464fa72b43a1c91f9a8b', 'request_id': 'req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:25.321655 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4806bebe2c89464fa72b43a1c91f9a8b', 'request_id': 'req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:25.340919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] The token cache is being invalidate because project 4806bebe2c89464fa72b43a1c91f9a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:43:25.343440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:25.343440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:25.348260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1114/2231] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:43:25 2022] DELETE /identity/v3/projects/4806bebe2c89464fa72b43a1c91f9a8b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:25.348664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cace9b-483d-4653-80d2-7491be7f3a35 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:25.364368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cace9b-483d-4653-80d2-7491be7f3a35 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:25.429033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24101804-ba1d-4637-b39f-c880c92c54cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:25.436875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24101804-ba1d-4637-b39f-c880c92c54cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:25.436875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24101804-ba1d-4637-b39f-c880c92c54cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:25.436875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24101804-ba1d-4637-b39f-c880c92c54cd admin admin] PATH_INFO: `/v3/users/5315912babe4420eb05d70c363dd7224` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:25.436875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24101804-ba1d-4637-b39f-c880c92c54cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5315912babe4420eb05d70c363dd7224)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:25.436875 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24101804-ba1d-4637-b39f-c880c92c54cd admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:25.637046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24101804-ba1d-4637-b39f-c880c92c54cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5315912babe4420eb05d70c363dd7224'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:25.717728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24101804-ba1d-4637-b39f-c880c92c54cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5315912babe4420eb05d70c363dd7224'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:25.733716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24101804-ba1d-4637-b39f-c880c92c54cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5315912babe4420eb05d70c363dd7224'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:25.734587 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24101804-ba1d-4637-b39f-c880c92c54cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5315912babe4420eb05d70c363dd7224'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:25.743134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-24101804-ba1d-4637-b39f-c880c92c54cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5315912babe4420eb05d70c363dd7224'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:25.750586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1118/2232] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:43:25 2022] DELETE /identity/v3/users/5315912babe4420eb05d70c363dd7224 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:26.190556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:26.191977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:26.212948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a6587-d7f6-4bb0-b58f-6486c9a1205c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:26.305378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:26.306064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:26.306438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:26.306722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] PATH_INFO: `/v3/projects/aa930c5eab444ae7aee813d6acc9b0bd` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:26.307979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa930c5eab444ae7aee813d6acc9b0bd)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:26.309906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:26.320020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:26.455500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa930c5eab444ae7aee813d6acc9b0bd', 'request_id': 'req-17cdf4d6-ce39-45cf-b45b-a64470d52156'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:26.465831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa930c5eab444ae7aee813d6acc9b0bd', 'request_id': 'req-17cdf4d6-ce39-45cf-b45b-a64470d52156'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:26.580874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa930c5eab444ae7aee813d6acc9b0bd', 'request_id': 'req-17cdf4d6-ce39-45cf-b45b-a64470d52156'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:26.597294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] The token cache is being invalidate because project aa930c5eab444ae7aee813d6acc9b0bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:43:26.597727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:26.598605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-17cdf4d6-ce39-45cf-b45b-a64470d52156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:26.600478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1115/2233] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:43:26 2022] DELETE /identity/v3/projects/aa930c5eab444ae7aee813d6acc9b0bd => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:26.605230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdde62-71f9-43c1-8ce1-ad81dd9d4533 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:26.605745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdde62-71f9-43c1-8ce1-ad81dd9d4533 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:26.605987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdde62-71f9-43c1-8ce1-ad81dd9d4533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:26.633254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-51cdde62-71f9-43c1-8ce1-ad81dd9d4533 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:26.634794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-51cdde62-71f9-43c1-8ce1-ad81dd9d4533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:26.671152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:26.700164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1119/2234] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:26.707760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ed5ca-0fca-4727-bb9c-420c53447279 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:26.708085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ed5ca-0fca-4727-bb9c-420c53447279 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:26.708364 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ed5ca-0fca-4727-bb9c-420c53447279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:26.735162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-da2ed5ca-0fca-4727-bb9c-420c53447279 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:26.736396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-da2ed5ca-0fca-4727-bb9c-420c53447279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:26.750492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1116/2235] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:43:26.755873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdde62-71f9-43c1-8ce1-ad81dd9d4533 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:26.767855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdde62-71f9-43c1-8ce1-ad81dd9d4533 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:26.772862 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861efc6-0fdf-4f9d-a8b4-a1cda986cb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:26.773498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861efc6-0fdf-4f9d-a8b4-a1cda986cb2c admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:26.773947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861efc6-0fdf-4f9d-a8b4-a1cda986cb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:26.774192 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861efc6-0fdf-4f9d-a8b4-a1cda986cb2c admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:26.774807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861efc6-0fdf-4f9d-a8b4-a1cda986cb2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:43:26.775123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861efc6-0fdf-4f9d-a8b4-a1cda986cb2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:26.777149 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861efc6-0fdf-4f9d-a8b4-a1cda986cb2c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:26.791180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1120/2236] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:43:26.806833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ed5ca-0fca-4727-bb9c-420c53447279 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:26.821449 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ed5ca-0fca-4727-bb9c-420c53447279 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:26.826764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04ffd9-d417-4f7a-873d-e28d45cb2954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:26.827757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04ffd9-d417-4f7a-873d-e28d45cb2954 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:26.827757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04ffd9-d417-4f7a-873d-e28d45cb2954 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:26.828594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04ffd9-d417-4f7a-873d-e28d45cb2954 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:26.829971 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04ffd9-d417-4f7a-873d-e28d45cb2954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:26.834212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04ffd9-d417-4f7a-873d-e28d45cb2954 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:26.945424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1117/2237] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:43:26 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:43:26.949451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861efc6-0fdf-4f9d-a8b4-a1cda986cb2c admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:26.962834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861efc6-0fdf-4f9d-a8b4-a1cda986cb2c admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:27.027952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08143c5-a91e-4ebb-99e3-718a382b0a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:27.028993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08143c5-a91e-4ebb-99e3-718a382b0a86 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:27.029451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08143c5-a91e-4ebb-99e3-718a382b0a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:27.029893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08143c5-a91e-4ebb-99e3-718a382b0a86 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:27.030749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08143c5-a91e-4ebb-99e3-718a382b0a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:27.033076 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08143c5-a91e-4ebb-99e3-718a382b0a86 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:27.149643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1121/2238] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:43:26 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:43:27.154189 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04ffd9-d417-4f7a-873d-e28d45cb2954 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:27.171493 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04ffd9-d417-4f7a-873d-e28d45cb2954 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:27.178020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7364b7-9fd4-4d3c-af55-49c5e51211d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:27.178930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7364b7-9fd4-4d3c-af55-49c5e51211d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:27.179222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7364b7-9fd4-4d3c-af55-49c5e51211d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:27.179668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7364b7-9fd4-4d3c-af55-49c5e51211d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:27.180652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7364b7-9fd4-4d3c-af55-49c5e51211d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:27.182295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7364b7-9fd4-4d3c-af55-49c5e51211d2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:27.201026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1118/2239] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:43:27.202066 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08143c5-a91e-4ebb-99e3-718a382b0a86 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:27.213465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08143c5-a91e-4ebb-99e3-718a382b0a86 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:27.218537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f10ce-9322-47f9-a4c9-94b4d348f1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:27.219254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f10ce-9322-47f9-a4c9-94b4d348f1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:27.219903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f10ce-9322-47f9-a4c9-94b4d348f1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:27.220272 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f10ce-9322-47f9-a4c9-94b4d348f1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:27.221090 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f10ce-9322-47f9-a4c9-94b4d348f1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:27.222843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f10ce-9322-47f9-a4c9-94b4d348f1f4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:27.238043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1122/2240] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:43:27.239764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7364b7-9fd4-4d3c-af55-49c5e51211d2 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:27.253974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7364b7-9fd4-4d3c-af55-49c5e51211d2 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:27.259946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c2f00-eef3-4b53-951a-cbb36ec123b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:27.260693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c2f00-eef3-4b53-951a-cbb36ec123b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:27.260912 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c2f00-eef3-4b53-951a-cbb36ec123b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:27.261196 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c2f00-eef3-4b53-951a-cbb36ec123b7 admin admin] PATH_INFO: `/v3/projects/c050bc57f7e14f44b1f28ae799283d2b/users/2f8ef071bc864e07a85a4bf806c26e06/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:27.279721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c2f00-eef3-4b53-951a-cbb36ec123b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c050bc57f7e14f44b1f28ae799283d2b, user_id=2f8ef071bc864e07a85a4bf806c26e06, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:27.282713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c2f00-eef3-4b53-951a-cbb36ec123b7 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:27.365761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1119/2241] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:43:27 2022] PUT /identity/v3/projects/c050bc57f7e14f44b1f28ae799283d2b/users/2f8ef071bc864e07a85a4bf806c26e06/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:43:27.378195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0500da-0fd0-416a-b087-16ba6498d7b7 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:27.378867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0500da-0fd0-416a-b087-16ba6498d7b7 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:27.379353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0500da-0fd0-416a-b087-16ba6498d7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:27.433447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bc0500da-0fd0-416a-b087-16ba6498d7b7 None None] MFA Rules not processed for user `2f8ef071bc864e07a85a4bf806c26e06`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:27.435038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bc0500da-0fd0-416a-b087-16ba6498d7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:27.527376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1123/2242] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:27.529023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c2f00-eef3-4b53-951a-cbb36ec123b7 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:27.617993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c2f00-eef3-4b53-951a-cbb36ec123b7 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:27.635420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43140562-da54-4c7e-bb8b-954d5dd4bde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:27.637218 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43140562-da54-4c7e-bb8b-954d5dd4bde5 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:27.637218 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43140562-da54-4c7e-bb8b-954d5dd4bde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:27.637218 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43140562-da54-4c7e-bb8b-954d5dd4bde5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:27.637575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43140562-da54-4c7e-bb8b-954d5dd4bde5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:27.639500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43140562-da54-4c7e-bb8b-954d5dd4bde5 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:27.764165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1120/2243] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:43:27 2022] POST /identity/v3/projects => generated 375 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:43:27.765370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0500da-0fd0-416a-b087-16ba6498d7b7 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:27.779783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0500da-0fd0-416a-b087-16ba6498d7b7 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:27.818486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:27.853344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f7cf0-cefd-4964-8f42-7a8cd4875582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:27.853970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f7cf0-cefd-4964-8f42-7a8cd4875582 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:27.854324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f7cf0-cefd-4964-8f42-7a8cd4875582 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:27.854629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f7cf0-cefd-4964-8f42-7a8cd4875582 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:27.855309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f7cf0-cefd-4964-8f42-7a8cd4875582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:27.857211 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f7cf0-cefd-4964-8f42-7a8cd4875582 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:27.976781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1124/2244] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:43:27 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:43:27.977682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43140562-da54-4c7e-bb8b-954d5dd4bde5 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:27.996736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43140562-da54-4c7e-bb8b-954d5dd4bde5 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:28.007719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee59cb-a9d5-4d50-9377-19772aaafc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:28.008665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee59cb-a9d5-4d50-9377-19772aaafc27 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:28.009412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee59cb-a9d5-4d50-9377-19772aaafc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:28.009763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee59cb-a9d5-4d50-9377-19772aaafc27 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:28.010598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee59cb-a9d5-4d50-9377-19772aaafc27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:28.012262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee59cb-a9d5-4d50-9377-19772aaafc27 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:28.024814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1121/2245] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:43:28.029369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f7cf0-cefd-4964-8f42-7a8cd4875582 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:28.044169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f7cf0-cefd-4964-8f42-7a8cd4875582 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:28.049311 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c83e35-97f2-4cb6-9a7f-96f95c365d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:28.050101 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c83e35-97f2-4cb6-9a7f-96f95c365d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:28.050456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c83e35-97f2-4cb6-9a7f-96f95c365d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:28.050740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c83e35-97f2-4cb6-9a7f-96f95c365d28 admin admin] PATH_INFO: `/v3/projects/93412eeb2765400cbb57e9c167928dda/users/870aeabaeb604dc5ac28045b0a0defab/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:28.073123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c83e35-97f2-4cb6-9a7f-96f95c365d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93412eeb2765400cbb57e9c167928dda, user_id=870aeabaeb604dc5ac28045b0a0defab, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:28.075940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c83e35-97f2-4cb6-9a7f-96f95c365d28 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:28.200679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1125/2246] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:43:28 2022] PUT /identity/v3/projects/93412eeb2765400cbb57e9c167928dda/users/870aeabaeb604dc5ac28045b0a0defab/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:28.206555 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee59cb-a9d5-4d50-9377-19772aaafc27 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:28.226217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee59cb-a9d5-4d50-9377-19772aaafc27 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:28.244439 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:28.349805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e875e8-b4a9-4626-b662-e5d96a2def10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:28.352003 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e875e8-b4a9-4626-b662-e5d96a2def10 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:28.352656 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e875e8-b4a9-4626-b662-e5d96a2def10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:28.353427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e875e8-b4a9-4626-b662-e5d96a2def10 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:28.354600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e875e8-b4a9-4626-b662-e5d96a2def10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:28.356771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e875e8-b4a9-4626-b662-e5d96a2def10 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:28.370970 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1122/2247] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:43:28.375552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c83e35-97f2-4cb6-9a7f-96f95c365d28 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:28.386928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c83e35-97f2-4cb6-9a7f-96f95c365d28 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:28.391855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42277f3f-4be4-4541-85ec-371d96ca7bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:28.392632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42277f3f-4be4-4541-85ec-371d96ca7bef admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:28.392980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42277f3f-4be4-4541-85ec-371d96ca7bef admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:28.393321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42277f3f-4be4-4541-85ec-371d96ca7bef admin admin] PATH_INFO: `/v3/projects/93412eeb2765400cbb57e9c167928dda/users/870aeabaeb604dc5ac28045b0a0defab/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:28.395989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42277f3f-4be4-4541-85ec-371d96ca7bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=93412eeb2765400cbb57e9c167928dda, user_id=870aeabaeb604dc5ac28045b0a0defab, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:28.399013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42277f3f-4be4-4541-85ec-371d96ca7bef admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:28.410085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1126/2248] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:43:28 2022] PUT /identity/v3/projects/93412eeb2765400cbb57e9c167928dda/users/870aeabaeb604dc5ac28045b0a0defab/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:28.421118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43457ad-16b4-497e-8007-9d843163bcb4 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:28.421429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43457ad-16b4-497e-8007-9d843163bcb4 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:28.421701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43457ad-16b4-497e-8007-9d843163bcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:28.454388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a43457ad-16b4-497e-8007-9d843163bcb4 None None] MFA Rules not processed for user `870aeabaeb604dc5ac28045b0a0defab`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:28.455917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a43457ad-16b4-497e-8007-9d843163bcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:28.581814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1123/2249] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:28.591775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42277f3f-4be4-4541-85ec-371d96ca7bef admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:28.593568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-42277f3f-4be4-4541-85ec-371d96ca7bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:28.675906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42277f3f-4be4-4541-85ec-371d96ca7bef admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:28.680836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d56ccc-eeab-431f-9ce4-3b3a90f91eaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:28.682812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d56ccc-eeab-431f-9ce4-3b3a90f91eaf service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:28.682812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d56ccc-eeab-431f-9ce4-3b3a90f91eaf service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:28.682812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d56ccc-eeab-431f-9ce4-3b3a90f91eaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:28.784390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d56ccc-eeab-431f-9ce4-3b3a90f91eaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:28.786481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d56ccc-eeab-431f-9ce4-3b3a90f91eaf service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:28.797662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1127/2250] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:43:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:43:38.389967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43457ad-16b4-497e-8007-9d843163bcb4 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:38.399718 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43457ad-16b4-497e-8007-9d843163bcb4 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:38.403266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509c36a-1659-4aa8-a765-e6a0a38e8f0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:38.403867 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509c36a-1659-4aa8-a765-e6a0a38e8f0d service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:38.404150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509c36a-1659-4aa8-a765-e6a0a38e8f0d service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:38.404425 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509c36a-1659-4aa8-a765-e6a0a38e8f0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:38.413659 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509c36a-1659-4aa8-a765-e6a0a38e8f0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:38.415153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509c36a-1659-4aa8-a765-e6a0a38e8f0d service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:38.425663 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1124/2251] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:43:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:43:38.792761 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:39.397738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeff6b-c936-493e-af66-d417c690d849 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:39.398025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeff6b-c936-493e-af66-d417c690d849 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:39.398309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeff6b-c936-493e-af66-d417c690d849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:39.417354 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c3eeff6b-c936-493e-af66-d417c690d849 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:39.418975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c3eeff6b-c936-493e-af66-d417c690d849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:39.474699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1128/2252] 10.222.0.73 () {58 vars in 967 bytes} [Thu Aug 4 21:43:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:39.487959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509c36a-1659-4aa8-a765-e6a0a38e8f0d service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:39.489412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-9509c36a-1659-4aa8-a765-e6a0a38e8f0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:39.566302 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509c36a-1659-4aa8-a765-e6a0a38e8f0d service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:39.571840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad7c8f-2b44-4264-9447-9560f69bfcb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:39.573866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad7c8f-2b44-4264-9447-9560f69bfcb4 service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:39.573866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad7c8f-2b44-4264-9447-9560f69bfcb4 service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:39.573866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad7c8f-2b44-4264-9447-9560f69bfcb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:39.580972 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad7c8f-2b44-4264-9447-9560f69bfcb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:39.582634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad7c8f-2b44-4264-9447-9560f69bfcb4 service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:39.591484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1125/2253] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:43:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:43:46.895264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeff6b-c936-493e-af66-d417c690d849 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:46.897393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c3eeff6b-c936-493e-af66-d417c690d849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:46.971649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeff6b-c936-493e-af66-d417c690d849 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:46.976856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8355b8-637b-4857-bd68-273e13ceab23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:46.977533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8355b8-637b-4857-bd68-273e13ceab23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:46.977816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8355b8-637b-4857-bd68-273e13ceab23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:46.978120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8355b8-637b-4857-bd68-273e13ceab23 admin admin] PATH_INFO: `/v3/users/db4a45e9f40740c99cc49d9f2978332d` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:46.979285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8355b8-637b-4857-bd68-273e13ceab23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db4a45e9f40740c99cc49d9f2978332d)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:46.981196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8355b8-637b-4857-bd68-273e13ceab23 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:47.184589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8a8355b8-637b-4857-bd68-273e13ceab23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db4a45e9f40740c99cc49d9f2978332d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:47.283173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8a8355b8-637b-4857-bd68-273e13ceab23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db4a45e9f40740c99cc49d9f2978332d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:47.308593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8a8355b8-637b-4857-bd68-273e13ceab23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db4a45e9f40740c99cc49d9f2978332d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:47.309529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8a8355b8-637b-4857-bd68-273e13ceab23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db4a45e9f40740c99cc49d9f2978332d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:47.317244 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-8a8355b8-637b-4857-bd68-273e13ceab23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db4a45e9f40740c99cc49d9f2978332d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:47.324941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1129/2254] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:43:46 2022] DELETE /identity/v3/users/db4a45e9f40740c99cc49d9f2978332d => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:47.575681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad7c8f-2b44-4264-9447-9560f69bfcb4 service swift] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:47.583525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:47.587982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad7c8f-2b44-4264-9447-9560f69bfcb4 service swift] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:47.646102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:47.646807 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:47.647089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:47.647362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] PATH_INFO: `/v3/projects/b37a8c7b37ac4f8d9711ac8cd0e37f02` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:47.648566 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b37a8c7b37ac4f8d9711ac8cd0e37f02)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:47.650514 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:47.752010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b37a8c7b37ac4f8d9711ac8cd0e37f02', 'request_id': 'req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:47.758411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b37a8c7b37ac4f8d9711ac8cd0e37f02', 'request_id': 'req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:47.819842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b37a8c7b37ac4f8d9711ac8cd0e37f02', 'request_id': 'req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:47.830454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] The token cache is being invalidate because project b37a8c7b37ac4f8d9711ac8cd0e37f02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:43:47.830743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:47.831428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14bc8d94-21c3-4678-b6d2-3d8dcc00f2c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:43:47.832958 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1126/2255] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:43:47 2022] DELETE /identity/v3/projects/b37a8c7b37ac4f8d9711ac8cd0e37f02 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:47.838104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe4897-4a0c-4c32-aae9-af43361f4541 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:47.838393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe4897-4a0c-4c32-aae9-af43361f4541 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:47.838668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe4897-4a0c-4c32-aae9-af43361f4541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:47.858475 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f0fe4897-4a0c-4c32-aae9-af43361f4541 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:47.859745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f0fe4897-4a0c-4c32-aae9-af43361f4541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:47.879310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:47.921486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1130/2256] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:47.929512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cdec3-43f6-4238-b957-386f3ee93b84 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:47.929805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cdec3-43f6-4238-b957-386f3ee93b84 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:47.930119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cdec3-43f6-4238-b957-386f3ee93b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:47.949808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-fb5cdec3-43f6-4238-b957-386f3ee93b84 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:47.951060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-fb5cdec3-43f6-4238-b957-386f3ee93b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:47.959910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1127/2257] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:47.964180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe4897-4a0c-4c32-aae9-af43361f4541 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:47.974585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe4897-4a0c-4c32-aae9-af43361f4541 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:47.979233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d6f72-7926-4266-817a-b2951f0b5aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:47.979918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6f72-7926-4266-817a-b2951f0b5aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:47.980199 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6f72-7926-4266-817a-b2951f0b5aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:47.980470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6f72-7926-4266-817a-b2951f0b5aa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:47.981168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d6f72-7926-4266-817a-b2951f0b5aa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:43:47.981493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d6f72-7926-4266-817a-b2951f0b5aa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:47.983164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d6f72-7926-4266-817a-b2951f0b5aa9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:47.995213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1131/2258] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:43:47.999611 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5cdec3-43f6-4238-b957-386f3ee93b84 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:48.009117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5cdec3-43f6-4238-b957-386f3ee93b84 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:48.013867 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2f9a0-cf1d-4886-aa15-e2421088a613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:48.014515 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2f9a0-cf1d-4886-aa15-e2421088a613 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:48.014799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2f9a0-cf1d-4886-aa15-e2421088a613 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:48.015074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2f9a0-cf1d-4886-aa15-e2421088a613 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:48.015733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2f9a0-cf1d-4886-aa15-e2421088a613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:48.017633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2f9a0-cf1d-4886-aa15-e2421088a613 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:48.079641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1128/2259] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:43:47 2022] POST /identity/v3/projects => generated 365 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:43:48.084310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d6f72-7926-4266-817a-b2951f0b5aa9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:48.094830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d6f72-7926-4266-817a-b2951f0b5aa9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:48.150266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b262b2b-302f-4a6f-868c-28f26ed20aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:48.150876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b262b2b-302f-4a6f-868c-28f26ed20aab admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:48.151158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b262b2b-302f-4a6f-868c-28f26ed20aab admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:48.151433 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b262b2b-302f-4a6f-868c-28f26ed20aab admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:48.152090 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b262b2b-302f-4a6f-868c-28f26ed20aab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:48.153980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b262b2b-302f-4a6f-868c-28f26ed20aab admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:48.221149 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1132/2260] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:43:48 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:43:48.226136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2f9a0-cf1d-4886-aa15-e2421088a613 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:48.236652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2f9a0-cf1d-4886-aa15-e2421088a613 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:48.241358 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c52b67-3c9a-4062-82e7-c04e5ea92514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:48.241957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c52b67-3c9a-4062-82e7-c04e5ea92514 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:48.242261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c52b67-3c9a-4062-82e7-c04e5ea92514 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:48.242533 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c52b67-3c9a-4062-82e7-c04e5ea92514 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:48.243214 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c52b67-3c9a-4062-82e7-c04e5ea92514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:48.245065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c52b67-3c9a-4062-82e7-c04e5ea92514 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:48.254801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1129/2261] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:43:48.259358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b262b2b-302f-4a6f-868c-28f26ed20aab admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:48.269193 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b262b2b-302f-4a6f-868c-28f26ed20aab admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:48.274714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1726ef3-2063-4671-84e3-349eabe9abdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:48.275309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1726ef3-2063-4671-84e3-349eabe9abdd admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:48.275589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1726ef3-2063-4671-84e3-349eabe9abdd admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:48.275858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1726ef3-2063-4671-84e3-349eabe9abdd admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:48.276570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1726ef3-2063-4671-84e3-349eabe9abdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:48.278197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1726ef3-2063-4671-84e3-349eabe9abdd admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:48.289133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1133/2262] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:43:48.293479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c52b67-3c9a-4062-82e7-c04e5ea92514 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:48.304024 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c52b67-3c9a-4062-82e7-c04e5ea92514 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:48.308677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758a8e7-ecc6-4b77-9234-8640d4537b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:48.309324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758a8e7-ecc6-4b77-9234-8640d4537b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:48.309609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758a8e7-ecc6-4b77-9234-8640d4537b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:48.309885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758a8e7-ecc6-4b77-9234-8640d4537b68 admin admin] PATH_INFO: `/v3/projects/a8170a9eef56402289850f880fe0dc44/users/df8864b104d7435882590db91b7e9f7f/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:48.324285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758a8e7-ecc6-4b77-9234-8640d4537b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8170a9eef56402289850f880fe0dc44, user_id=df8864b104d7435882590db91b7e9f7f, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:48.327105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758a8e7-ecc6-4b77-9234-8640d4537b68 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:48.432390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1130/2263] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:43:48 2022] PUT /identity/v3/projects/a8170a9eef56402289850f880fe0dc44/users/df8864b104d7435882590db91b7e9f7f/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:43:48.443425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4858b76-d56c-4220-b6ed-6c079a8ff58d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:48.443716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4858b76-d56c-4220-b6ed-6c079a8ff58d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:48.443994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4858b76-d56c-4220-b6ed-6c079a8ff58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:48.470862 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f4858b76-d56c-4220-b6ed-6c079a8ff58d None None] MFA Rules not processed for user `df8864b104d7435882590db91b7e9f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:43:48.472047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f4858b76-d56c-4220-b6ed-6c079a8ff58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:48.535301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1134/2264] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:43:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:43:48.546851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758a8e7-ecc6-4b77-9234-8640d4537b68 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:43:48.548565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0758a8e7-ecc6-4b77-9234-8640d4537b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:43:48.582351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:48.606520 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758a8e7-ecc6-4b77-9234-8640d4537b68 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:43:48.610843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92eaad-1c3b-43a4-8919-95a900feaf62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:43:48.611447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92eaad-1c3b-43a4-8919-95a900feaf62 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:43:48.611732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92eaad-1c3b-43a4-8919-95a900feaf62 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:43:48.612011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92eaad-1c3b-43a4-8919-95a900feaf62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:43:48.620746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92eaad-1c3b-43a4-8919-95a900feaf62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:43:48.622206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92eaad-1c3b-43a4-8919-95a900feaf62 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:43:48.631457 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1131/2265] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:43:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:43:58.532310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:43:58.666711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=12 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:20.680523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4858b76-d56c-4220-b6ed-6c079a8ff58d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:20.681951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f4858b76-d56c-4220-b6ed-6c079a8ff58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:20.747123 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4858b76-d56c-4220-b6ed-6c079a8ff58d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:20.751666 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:20.752297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:20.752575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:20.752845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] PATH_INFO: `/v3/users/2f8ef071bc864e07a85a4bf806c26e06` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:20.753994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8ef071bc864e07a85a4bf806c26e06)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:20.755906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:20.944358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8ef071bc864e07a85a4bf806c26e06'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:21.009443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8ef071bc864e07a85a4bf806c26e06'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:21.023194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8ef071bc864e07a85a4bf806c26e06'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:21.023974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8ef071bc864e07a85a4bf806c26e06'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:21.032046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8ef071bc864e07a85a4bf806c26e06'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:21.038438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1135/2266] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:44:20 2022] DELETE /identity/v3/users/2f8ef071bc864e07a85a4bf806c26e06 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:21.043117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e1668-53c6-4200-82bb-8f00945b15d9 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:21.043679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e1668-53c6-4200-82bb-8f00945b15d9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:21.043679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e1668-53c6-4200-82bb-8f00945b15d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:21.064646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-511e1668-53c6-4200-82bb-8f00945b15d9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:21.065984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-511e1668-53c6-4200-82bb-8f00945b15d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:21.132993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1132/2267] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:21.146567 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:21.148051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:21.208788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdabf86-f73c-4310-855b-fa6f7ba115d1 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:21.212568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57973e92-2855-4c0f-a557-0ff1f034cd2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:21.213183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57973e92-2855-4c0f-a557-0ff1f034cd2c service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:21.213474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57973e92-2855-4c0f-a557-0ff1f034cd2c service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:21.213743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57973e92-2855-4c0f-a557-0ff1f034cd2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:21.223396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57973e92-2855-4c0f-a557-0ff1f034cd2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:21.225134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57973e92-2855-4c0f-a557-0ff1f034cd2c service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:21.235625 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1136/2268] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:44:21.615579 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e1668-53c6-4200-82bb-8f00945b15d9 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:21.617013 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-511e1668-53c6-4200-82bb-8f00945b15d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:21.629990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e1668-53c6-4200-82bb-8f00945b15d9 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:21.635256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:21.635914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:21.636240 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:21.636519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] PATH_INFO: `/v3/projects/c050bc57f7e14f44b1f28ae799283d2b` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:21.637653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c050bc57f7e14f44b1f28ae799283d2b)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:21.640265 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:21.736997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c050bc57f7e14f44b1f28ae799283d2b', 'request_id': 'req-25ea2f91-6261-4f97-92ea-613b9d56d6b8'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:21.743055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c050bc57f7e14f44b1f28ae799283d2b', 'request_id': 'req-25ea2f91-6261-4f97-92ea-613b9d56d6b8'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:21.849177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c050bc57f7e14f44b1f28ae799283d2b', 'request_id': 'req-25ea2f91-6261-4f97-92ea-613b9d56d6b8'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:21.860616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] The token cache is being invalidate because project c050bc57f7e14f44b1f28ae799283d2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:44:21.860915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:21.861693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:21.863318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1133/2269] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:44:21 2022] DELETE /identity/v3/projects/c050bc57f7e14f44b1f28ae799283d2b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:21.867460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57973e92-2855-4c0f-a557-0ff1f034cd2c service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:21.878954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57973e92-2855-4c0f-a557-0ff1f034cd2c service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:21.898298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:21.937458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c25d8-5bc3-4cdc-8d25-5b66aead6c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:21.938293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c25d8-5bc3-4cdc-8d25-5b66aead6c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:21.938584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c25d8-5bc3-4cdc-8d25-5b66aead6c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:21.938858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c25d8-5bc3-4cdc-8d25-5b66aead6c92 admin admin] PATH_INFO: `/v3/users/870aeabaeb604dc5ac28045b0a0defab` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:21.940065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c25d8-5bc3-4cdc-8d25-5b66aead6c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=870aeabaeb604dc5ac28045b0a0defab)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:21.942034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c25d8-5bc3-4cdc-8d25-5b66aead6c92 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:22.126083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-438c25d8-5bc3-4cdc-8d25-5b66aead6c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870aeabaeb604dc5ac28045b0a0defab'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:22.186513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-438c25d8-5bc3-4cdc-8d25-5b66aead6c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870aeabaeb604dc5ac28045b0a0defab'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:22.198965 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-438c25d8-5bc3-4cdc-8d25-5b66aead6c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870aeabaeb604dc5ac28045b0a0defab'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:22.199607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-438c25d8-5bc3-4cdc-8d25-5b66aead6c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870aeabaeb604dc5ac28045b0a0defab'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:22.208091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-438c25d8-5bc3-4cdc-8d25-5b66aead6c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870aeabaeb604dc5ac28045b0a0defab'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:22.214410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1137/2270] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:44:21 2022] DELETE /identity/v3/users/870aeabaeb604dc5ac28045b0a0defab => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:44:22.628275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:22.629803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:22.644167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea2f91-6261-4f97-92ea-613b9d56d6b8 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:22.703115 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:22.703754 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:22.704037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:22.704312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] PATH_INFO: `/v3/projects/93412eeb2765400cbb57e9c167928dda` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:22.705462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93412eeb2765400cbb57e9c167928dda)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:22.707409 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:22.805295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:22.821607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93412eeb2765400cbb57e9c167928dda', 'request_id': 'req-6d1466c2-0385-4887-b861-5bced3fd1319'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:22.828218 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93412eeb2765400cbb57e9c167928dda', 'request_id': 'req-6d1466c2-0385-4887-b861-5bced3fd1319'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:22.889538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93412eeb2765400cbb57e9c167928dda', 'request_id': 'req-6d1466c2-0385-4887-b861-5bced3fd1319'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:22.900800 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] The token cache is being invalidate because project 93412eeb2765400cbb57e9c167928dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:44:22.901086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:22.901707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6d1466c2-0385-4887-b861-5bced3fd1319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:22.904026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1134/2271] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:44:22 2022] DELETE /identity/v3/projects/93412eeb2765400cbb57e9c167928dda => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:22.908983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a438f42-88d4-42c4-aa12-f8fdbb47930f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:22.913886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a438f42-88d4-42c4-aa12-f8fdbb47930f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:22.913886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a438f42-88d4-42c4-aa12-f8fdbb47930f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:22.930917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-4a438f42-88d4-42c4-aa12-f8fdbb47930f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:22.932171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4a438f42-88d4-42c4-aa12-f8fdbb47930f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:22.990040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1138/2272] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:22.998507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4371b2b-b8f5-4beb-802a-eab823393460 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:22.998827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4371b2b-b8f5-4beb-802a-eab823393460 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:22.999151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4371b2b-b8f5-4beb-802a-eab823393460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:23.020632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-f4371b2b-b8f5-4beb-802a-eab823393460 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:23.021967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f4371b2b-b8f5-4beb-802a-eab823393460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:23.030669 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1135/2273] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:23.034814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a438f42-88d4-42c4-aa12-f8fdbb47930f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:23.044320 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a438f42-88d4-42c4-aa12-f8fdbb47930f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:23.048899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e67e1-1959-4409-9309-5bf371f6d6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:23.049495 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e67e1-1959-4409-9309-5bf371f6d6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:23.049777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e67e1-1959-4409-9309-5bf371f6d6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:23.050075 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e67e1-1959-4409-9309-5bf371f6d6d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:23.050765 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e67e1-1959-4409-9309-5bf371f6d6d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:44:23.051073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e67e1-1959-4409-9309-5bf371f6d6d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:23.052647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e67e1-1959-4409-9309-5bf371f6d6d1 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:23.064024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1139/2274] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:44:23.068094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4371b2b-b8f5-4beb-802a-eab823393460 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:23.077936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4371b2b-b8f5-4beb-802a-eab823393460 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:23.082677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cd7cb-85bd-4eb0-bc8b-e59623f002ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:23.083280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cd7cb-85bd-4eb0-bc8b-e59623f002ea admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:23.083565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cd7cb-85bd-4eb0-bc8b-e59623f002ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:23.083842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cd7cb-85bd-4eb0-bc8b-e59623f002ea admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:23.084525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cd7cb-85bd-4eb0-bc8b-e59623f002ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:23.086454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cd7cb-85bd-4eb0-bc8b-e59623f002ea admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:23.176792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1136/2275] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:44:23 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:44:23.180942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e67e1-1959-4409-9309-5bf371f6d6d1 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:23.192083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e67e1-1959-4409-9309-5bf371f6d6d1 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:23.218655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:23.249054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f731c-1d0b-4dfc-b67b-9569429e6bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:23.249671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f731c-1d0b-4dfc-b67b-9569429e6bdd admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:23.249953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f731c-1d0b-4dfc-b67b-9569429e6bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:23.250274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f731c-1d0b-4dfc-b67b-9569429e6bdd admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:23.250930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f731c-1d0b-4dfc-b67b-9569429e6bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:23.252825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f731c-1d0b-4dfc-b67b-9569429e6bdd admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:23.322307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1140/2276] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:44:23 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:44:23.326542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cd7cb-85bd-4eb0-bc8b-e59623f002ea admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:23.336748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cd7cb-85bd-4eb0-bc8b-e59623f002ea admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:23.342197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c109b3-6d31-44f7-83fd-442a0b6ada70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:23.342802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c109b3-6d31-44f7-83fd-442a0b6ada70 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:23.343085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c109b3-6d31-44f7-83fd-442a0b6ada70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:23.343358 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c109b3-6d31-44f7-83fd-442a0b6ada70 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:23.344084 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c109b3-6d31-44f7-83fd-442a0b6ada70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:23.345706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c109b3-6d31-44f7-83fd-442a0b6ada70 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:23.355762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1137/2277] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:44:23.360364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f731c-1d0b-4dfc-b67b-9569429e6bdd admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:23.370594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f731c-1d0b-4dfc-b67b-9569429e6bdd admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:23.375443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d27c1-7ff4-4f63-90bd-e8a0ac4885d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:23.376035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d27c1-7ff4-4f63-90bd-e8a0ac4885d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:23.376314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d27c1-7ff4-4f63-90bd-e8a0ac4885d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:23.376611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d27c1-7ff4-4f63-90bd-e8a0ac4885d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:23.377326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d27c1-7ff4-4f63-90bd-e8a0ac4885d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:23.378976 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d27c1-7ff4-4f63-90bd-e8a0ac4885d6 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:23.394245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1141/2278] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:23.395608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c109b3-6d31-44f7-83fd-442a0b6ada70 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:23.407891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c109b3-6d31-44f7-83fd-442a0b6ada70 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:23.413272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f212ec-6a5e-44bb-a5fe-c95d247aa108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:23.414172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f212ec-6a5e-44bb-a5fe-c95d247aa108 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:23.414456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f212ec-6a5e-44bb-a5fe-c95d247aa108 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:23.414782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f212ec-6a5e-44bb-a5fe-c95d247aa108 admin admin] PATH_INFO: `/v3/projects/3ad6a9b8a9084495838e2356e0766254/users/59e4973d289b46b9afd25e54dfb23665/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:23.437644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f212ec-6a5e-44bb-a5fe-c95d247aa108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ad6a9b8a9084495838e2356e0766254, user_id=59e4973d289b46b9afd25e54dfb23665, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:23.440329 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f212ec-6a5e-44bb-a5fe-c95d247aa108 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:23.536764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1138/2279] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:44:23 2022] PUT /identity/v3/projects/3ad6a9b8a9084495838e2356e0766254/users/59e4973d289b46b9afd25e54dfb23665/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:44:23.548092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9298d-81a5-489b-aa21-f669b0046e3e None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:23.548383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9298d-81a5-489b-aa21-f669b0046e3e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:23.548659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9298d-81a5-489b-aa21-f669b0046e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:23.576831 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-86e9298d-81a5-489b-aa21-f669b0046e3e None None] MFA Rules not processed for user `59e4973d289b46b9afd25e54dfb23665`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:23.578663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-86e9298d-81a5-489b-aa21-f669b0046e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:23.641533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1142/2280] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:44:23.646213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f212ec-6a5e-44bb-a5fe-c95d247aa108 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:23.706757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f212ec-6a5e-44bb-a5fe-c95d247aa108 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:23.711434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab0fa6-1428-4f2e-a477-8d0d2adc400e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:23.712039 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab0fa6-1428-4f2e-a477-8d0d2adc400e admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:23.712322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab0fa6-1428-4f2e-a477-8d0d2adc400e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:23.712596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab0fa6-1428-4f2e-a477-8d0d2adc400e admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:23.713254 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab0fa6-1428-4f2e-a477-8d0d2adc400e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:23.715173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab0fa6-1428-4f2e-a477-8d0d2adc400e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:23.828612 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1139/2281] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:44:23 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:44:23.832640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9298d-81a5-489b-aa21-f669b0046e3e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:23.843560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9298d-81a5-489b-aa21-f669b0046e3e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:23.899583 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3b268-3cd0-44f6-b53e-fa48713a24bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:23.900113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3b268-3cd0-44f6-b53e-fa48713a24bf admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:23.900391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3b268-3cd0-44f6-b53e-fa48713a24bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:23.900661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3b268-3cd0-44f6-b53e-fa48713a24bf admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:23.901314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3b268-3cd0-44f6-b53e-fa48713a24bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:23.903255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3b268-3cd0-44f6-b53e-fa48713a24bf admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:23.992143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1143/2282] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:44:23 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:44:23.996972 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab0fa6-1428-4f2e-a477-8d0d2adc400e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:24.007935 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab0fa6-1428-4f2e-a477-8d0d2adc400e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:24.012849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c03e5-91a7-4cb0-8ea2-bc06caa5d72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:24.013448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c03e5-91a7-4cb0-8ea2-bc06caa5d72c admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.013732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c03e5-91a7-4cb0-8ea2-bc06caa5d72c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.014029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c03e5-91a7-4cb0-8ea2-bc06caa5d72c admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.014722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c03e5-91a7-4cb0-8ea2-bc06caa5d72c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:24.016333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c03e5-91a7-4cb0-8ea2-bc06caa5d72c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:24.026944 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1140/2283] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:24.030993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3b268-3cd0-44f6-b53e-fa48713a24bf admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:24.040679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3b268-3cd0-44f6-b53e-fa48713a24bf admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:24.045825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3779731-dea4-426e-b643-5bc247761210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:24.046455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3779731-dea4-426e-b643-5bc247761210 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.046752 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3779731-dea4-426e-b643-5bc247761210 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.047000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3779731-dea4-426e-b643-5bc247761210 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.047690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3779731-dea4-426e-b643-5bc247761210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:24.049254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3779731-dea4-426e-b643-5bc247761210 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:24.058938 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1144/2284] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:24.063618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c03e5-91a7-4cb0-8ea2-bc06caa5d72c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:24.073570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c03e5-91a7-4cb0-8ea2-bc06caa5d72c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:24.078374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d4383-9a07-4021-a87e-176e840a6c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:24.079065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d4383-9a07-4021-a87e-176e840a6c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.079360 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d4383-9a07-4021-a87e-176e840a6c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.079631 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d4383-9a07-4021-a87e-176e840a6c6d admin admin] PATH_INFO: `/v3/projects/6ce6295191534315badd9d88e210dcad/users/2dbf29ade2e0462ea32023607b025859/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.095644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d4383-9a07-4021-a87e-176e840a6c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce6295191534315badd9d88e210dcad, user_id=2dbf29ade2e0462ea32023607b025859, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:24.098306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d4383-9a07-4021-a87e-176e840a6c6d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:24.165846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:24.173071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1141/2285] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:44:24 2022] PUT /identity/v3/projects/6ce6295191534315badd9d88e210dcad/users/2dbf29ade2e0462ea32023607b025859/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:24.184649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6064b0-b3a8-4223-9ffc-c7437bee5801 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.185165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6064b0-b3a8-4223-9ffc-c7437bee5801 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.185339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6064b0-b3a8-4223-9ffc-c7437bee5801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.202182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d4383-9a07-4021-a87e-176e840a6c6d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:24.204043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-995d4383-9a07-4021-a87e-176e840a6c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:24.214703 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-cd6064b0-b3a8-4223-9ffc-c7437bee5801 None None] MFA Rules not processed for user `2dbf29ade2e0462ea32023607b025859`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:24.216867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-cd6064b0-b3a8-4223-9ffc-c7437bee5801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:24.221996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:24.276581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d4383-9a07-4021-a87e-176e840a6c6d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:24.283025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbeeb3-465c-4462-ae47-b5f8819c2cc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:24.284843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1145/2286] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:24.284957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbeeb3-465c-4462-ae47-b5f8819c2cc9 service placement] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.284957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbeeb3-465c-4462-ae47-b5f8819c2cc9 service placement] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.284957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbeeb3-465c-4462-ae47-b5f8819c2cc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.286037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-46fbeeb3-465c-4462-ae47-b5f8819c2cc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:24.287911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6064b0-b3a8-4223-9ffc-c7437bee5801 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:24.298498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbeeb3-465c-4462-ae47-b5f8819c2cc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:24.300173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbeeb3-465c-4462-ae47-b5f8819c2cc9 service placement] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:24.311652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1142/2287] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:44:24.351852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6064b0-b3a8-4223-9ffc-c7437bee5801 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:24.356573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643e7b-aabf-4141-b880-6c38ba7ce414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:24.357258 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643e7b-aabf-4141-b880-6c38ba7ce414 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.358445 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643e7b-aabf-4141-b880-6c38ba7ce414 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.358723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643e7b-aabf-4141-b880-6c38ba7ce414 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.359384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b643e7b-aabf-4141-b880-6c38ba7ce414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:24.361294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b643e7b-aabf-4141-b880-6c38ba7ce414 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:24.439574 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1146/2288] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:44:24 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:44:24.443660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbeeb3-465c-4462-ae47-b5f8819c2cc9 service placement] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:24.455099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbeeb3-465c-4462-ae47-b5f8819c2cc9 service placement] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:24.515396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d6f68-4b38-4944-afd5-f1248d41060e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:24.516021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d6f68-4b38-4944-afd5-f1248d41060e admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.520020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d6f68-4b38-4944-afd5-f1248d41060e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.520020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d6f68-4b38-4944-afd5-f1248d41060e admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.520020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d6f68-4b38-4944-afd5-f1248d41060e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:24.520020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d6f68-4b38-4944-afd5-f1248d41060e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:24.618094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1143/2289] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:44:24 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:44:24.619610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643e7b-aabf-4141-b880-6c38ba7ce414 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:24.642599 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643e7b-aabf-4141-b880-6c38ba7ce414 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:24.647328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11174fe5-0da0-494a-a0e7-4022a88e611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:24.647949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11174fe5-0da0-494a-a0e7-4022a88e611b admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.648270 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11174fe5-0da0-494a-a0e7-4022a88e611b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.648691 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11174fe5-0da0-494a-a0e7-4022a88e611b admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.649233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11174fe5-0da0-494a-a0e7-4022a88e611b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:24.650847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11174fe5-0da0-494a-a0e7-4022a88e611b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:24.662920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1147/2290] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:24.667858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d6f68-4b38-4944-afd5-f1248d41060e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:24.678610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d6f68-4b38-4944-afd5-f1248d41060e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:24.684271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a675c-fdf6-47ab-b8d6-7623ac8a441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:24.685088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a675c-fdf6-47ab-b8d6-7623ac8a441b admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.685371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a675c-fdf6-47ab-b8d6-7623ac8a441b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.685739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a675c-fdf6-47ab-b8d6-7623ac8a441b admin admin] PATH_INFO: `/v3/projects/e711f1f9a75d4f95ab8906071a2a5196/users/166cc12fd48a4f7185a140c2f642a743/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.707316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a675c-fdf6-47ab-b8d6-7623ac8a441b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e711f1f9a75d4f95ab8906071a2a5196, user_id=166cc12fd48a4f7185a140c2f642a743, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:24.710567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a675c-fdf6-47ab-b8d6-7623ac8a441b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:24.810083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1144/2291] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:44:24 2022] PUT /identity/v3/projects/e711f1f9a75d4f95ab8906071a2a5196/users/166cc12fd48a4f7185a140c2f642a743/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:24.819113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11174fe5-0da0-494a-a0e7-4022a88e611b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:24.834081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11174fe5-0da0-494a-a0e7-4022a88e611b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:24.891435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580dedd-da9f-46b4-8b5e-fd97b99702b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:24.891435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580dedd-da9f-46b4-8b5e-fd97b99702b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.891435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580dedd-da9f-46b4-8b5e-fd97b99702b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.891435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580dedd-da9f-46b4-8b5e-fd97b99702b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.891835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580dedd-da9f-46b4-8b5e-fd97b99702b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:24.891835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580dedd-da9f-46b4-8b5e-fd97b99702b0 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:24.901680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1148/2292] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:44:24.906600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a675c-fdf6-47ab-b8d6-7623ac8a441b admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:24.916842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a675c-fdf6-47ab-b8d6-7623ac8a441b admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:24.922058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218080b0-150e-4522-ad4b-9f72b72040bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:24.922718 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218080b0-150e-4522-ad4b-9f72b72040bb admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.923000 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218080b0-150e-4522-ad4b-9f72b72040bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.923274 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218080b0-150e-4522-ad4b-9f72b72040bb admin admin] PATH_INFO: `/v3/projects/e711f1f9a75d4f95ab8906071a2a5196/users/166cc12fd48a4f7185a140c2f642a743/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.925225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218080b0-150e-4522-ad4b-9f72b72040bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e711f1f9a75d4f95ab8906071a2a5196, user_id=166cc12fd48a4f7185a140c2f642a743, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:24.927890 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218080b0-150e-4522-ad4b-9f72b72040bb admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:24.937944 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1145/2293] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:44:24 2022] PUT /identity/v3/projects/e711f1f9a75d4f95ab8906071a2a5196/users/166cc12fd48a4f7185a140c2f642a743/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:24.948075 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add92754-bf0b-49d7-92c0-109f96a4e953 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:24.948367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add92754-bf0b-49d7-92c0-109f96a4e953 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:24.948648 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add92754-bf0b-49d7-92c0-109f96a4e953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:24.977405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-add92754-bf0b-49d7-92c0-109f96a4e953 None None] MFA Rules not processed for user `166cc12fd48a4f7185a140c2f642a743`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:24.978616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-add92754-bf0b-49d7-92c0-109f96a4e953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:24.983168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:25.049143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1149/2294] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:25.059876 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218080b0-150e-4522-ad4b-9f72b72040bb admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:25.061882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-218080b0-150e-4522-ad4b-9f72b72040bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:25.108879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:25.120466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218080b0-150e-4522-ad4b-9f72b72040bb admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:25.124238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8e1dd-b098-43a7-a5ad-89fb39bfef23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:25.124849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8e1dd-b098-43a7-a5ad-89fb39bfef23 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:25.125132 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8e1dd-b098-43a7-a5ad-89fb39bfef23 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:25.125406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8e1dd-b098-43a7-a5ad-89fb39bfef23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:25.134913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8e1dd-b098-43a7-a5ad-89fb39bfef23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:25.136406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8e1dd-b098-43a7-a5ad-89fb39bfef23 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:25.146570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1146/2295] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:44:26.206966 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add92754-bf0b-49d7-92c0-109f96a4e953 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:26.208355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-add92754-bf0b-49d7-92c0-109f96a4e953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:26.221739 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add92754-bf0b-49d7-92c0-109f96a4e953 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:26.225701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7955166-e0aa-4194-bf3d-83869fe61632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:26.226563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955166-e0aa-4194-bf3d-83869fe61632 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:26.226845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955166-e0aa-4194-bf3d-83869fe61632 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:26.227114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955166-e0aa-4194-bf3d-83869fe61632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:26.286996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7955166-e0aa-4194-bf3d-83869fe61632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:26.288363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7955166-e0aa-4194-bf3d-83869fe61632 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:26.297889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1150/2296] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:44:33.390375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8e1dd-b098-43a7-a5ad-89fb39bfef23 service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:33.400217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8e1dd-b098-43a7-a5ad-89fb39bfef23 service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:33.404112 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442deb4-d9e1-4314-ade1-d9bf1c8a900b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:33.406091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442deb4-d9e1-4314-ade1-d9bf1c8a900b service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:33.406091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442deb4-d9e1-4314-ade1-d9bf1c8a900b service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:33.406091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442deb4-d9e1-4314-ade1-d9bf1c8a900b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:33.469841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442deb4-d9e1-4314-ade1-d9bf1c8a900b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:33.471370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442deb4-d9e1-4314-ade1-d9bf1c8a900b service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:33.480247 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1147/2297] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:44:36.293393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=55 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:37.465945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7955166-e0aa-4194-bf3d-83869fe61632 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:37.531325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7955166-e0aa-4194-bf3d-83869fe61632 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:37.536416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:37.537071 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:37.537353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:37.537629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] PATH_INFO: `/v3/users/59e4973d289b46b9afd25e54dfb23665` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:37.538929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59e4973d289b46b9afd25e54dfb23665)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:37.540878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:37.754517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59e4973d289b46b9afd25e54dfb23665'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:37.842358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59e4973d289b46b9afd25e54dfb23665'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:37.855901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59e4973d289b46b9afd25e54dfb23665'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:37.856814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59e4973d289b46b9afd25e54dfb23665'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:37.864213 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59e4973d289b46b9afd25e54dfb23665'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:37.870871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1151/2298] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:44:37 2022] DELETE /identity/v3/users/59e4973d289b46b9afd25e54dfb23665 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:37.876923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a6849-9f0a-487e-b594-190054f0506a None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:37.877942 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a6849-9f0a-487e-b594-190054f0506a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:37.878477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a6849-9f0a-487e-b594-190054f0506a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:37.898139 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-905a6849-9f0a-487e-b594-190054f0506a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:37.899388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-905a6849-9f0a-487e-b594-190054f0506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:37.961574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1148/2299] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:37.971465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:37.973529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:38.032277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085d63d-0d0e-4bcc-8619-65b6ec0e87d1 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:38.036589 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d4af7-50d2-473c-b41b-a279f6fd7bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:38.037195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4af7-50d2-473c-b41b-a279f6fd7bfb service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:38.037473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4af7-50d2-473c-b41b-a279f6fd7bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:38.037742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4af7-50d2-473c-b41b-a279f6fd7bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:38.047098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d4af7-50d2-473c-b41b-a279f6fd7bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:38.048558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d4af7-50d2-473c-b41b-a279f6fd7bfb service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:38.059644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1152/2300] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:44:38.455609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a6849-9f0a-487e-b594-190054f0506a None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:38.457879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-905a6849-9f0a-487e-b594-190054f0506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:38.472877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a6849-9f0a-487e-b594-190054f0506a None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:38.477786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:38.478540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:38.478814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:38.479083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] PATH_INFO: `/v3/projects/3ad6a9b8a9084495838e2356e0766254` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:38.480239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ad6a9b8a9084495838e2356e0766254)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:38.482212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:38.582460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ad6a9b8a9084495838e2356e0766254', 'request_id': 'req-3e51f84a-c810-45be-b26b-c555094690e6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:38.589782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ad6a9b8a9084495838e2356e0766254', 'request_id': 'req-3e51f84a-c810-45be-b26b-c555094690e6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:38.649021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ad6a9b8a9084495838e2356e0766254', 'request_id': 'req-3e51f84a-c810-45be-b26b-c555094690e6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:38.660063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] The token cache is being invalidate because project 3ad6a9b8a9084495838e2356e0766254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:44:38.660355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:38.661068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:38.662630 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1149/2301] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:44:38 2022] DELETE /identity/v3/projects/3ad6a9b8a9084495838e2356e0766254 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:38.667172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d4af7-50d2-473c-b41b-a279f6fd7bfb service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:38.677503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d4af7-50d2-473c-b41b-a279f6fd7bfb service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:38.696021 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:38.736121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:38.736784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:38.737062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:38.737341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] PATH_INFO: `/v3/users/2dbf29ade2e0462ea32023607b025859` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:38.738796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dbf29ade2e0462ea32023607b025859)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:38.741030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:38.943187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dbf29ade2e0462ea32023607b025859'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:39.023564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dbf29ade2e0462ea32023607b025859'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:39.038200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dbf29ade2e0462ea32023607b025859'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:39.038915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dbf29ade2e0462ea32023607b025859'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:39.051065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dbf29ade2e0462ea32023607b025859'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:39.057081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1153/2302] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:44:38 2022] DELETE /identity/v3/users/2dbf29ade2e0462ea32023607b025859 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:39.497767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:39.499450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:39.516318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51f84a-c810-45be-b26b-c555094690e6 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:39.520854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:39.578795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:39.579433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:39.579715 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:39.579989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] PATH_INFO: `/v3/projects/6ce6295191534315badd9d88e210dcad` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:39.581498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce6295191534315badd9d88e210dcad)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:39.583830 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:39.694314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce6295191534315badd9d88e210dcad', 'request_id': 'req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:39.701091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce6295191534315badd9d88e210dcad', 'request_id': 'req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:39.746699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce6295191534315badd9d88e210dcad', 'request_id': 'req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:39.758882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] The token cache is being invalidate because project 6ce6295191534315badd9d88e210dcad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:44:39.763280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:39.763280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:39.763280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1150/2303] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:44:39 2022] DELETE /identity/v3/projects/6ce6295191534315badd9d88e210dcad => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:39.766275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:39.767729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:39.781728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b6289-bf21-4915-a66d-22654fbf901e admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:39.839177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f46830-7d70-4c57-a59f-f64214931dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:39.839816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f46830-7d70-4c57-a59f-f64214931dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:39.840097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f46830-7d70-4c57-a59f-f64214931dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:39.840366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f46830-7d70-4c57-a59f-f64214931dc5 admin admin] PATH_INFO: `/v3/users/166cc12fd48a4f7185a140c2f642a743` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:39.841509 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f46830-7d70-4c57-a59f-f64214931dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=166cc12fd48a4f7185a140c2f642a743)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:39.843432 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f46830-7d70-4c57-a59f-f64214931dc5 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:39.998089 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:40.000677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-29f46830-7d70-4c57-a59f-f64214931dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166cc12fd48a4f7185a140c2f642a743'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:40.096518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-29f46830-7d70-4c57-a59f-f64214931dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166cc12fd48a4f7185a140c2f642a743'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:40.109210 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-29f46830-7d70-4c57-a59f-f64214931dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166cc12fd48a4f7185a140c2f642a743'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:40.109931 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-29f46830-7d70-4c57-a59f-f64214931dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166cc12fd48a4f7185a140c2f642a743'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:40.117188 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-29f46830-7d70-4c57-a59f-f64214931dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166cc12fd48a4f7185a140c2f642a743'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:40.124547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1154/2304] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:44:39 2022] DELETE /identity/v3/users/166cc12fd48a4f7185a140c2f642a743 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:44:40.662131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:40.663731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:40.678209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2bc90-e6ab-4054-9bf9-706b4c8239ae admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:40.736972 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:40.738265 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:40.738511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:40.738840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] PATH_INFO: `/v3/projects/e711f1f9a75d4f95ab8906071a2a5196` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:40.740581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=e711f1f9a75d4f95ab8906071a2a5196)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:40.742527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:40.855010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:40.862103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e711f1f9a75d4f95ab8906071a2a5196', 'request_id': 'req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:40.868982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e711f1f9a75d4f95ab8906071a2a5196', 'request_id': 'req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:40.922866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e711f1f9a75d4f95ab8906071a2a5196', 'request_id': 'req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:40.933874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] The token cache is being invalidate because project e711f1f9a75d4f95ab8906071a2a5196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:44:40.937974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:40.937974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-976cbb8b-48f0-44f2-ae01-bc0a33e67eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:40.937974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1151/2305] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:44:40 2022] DELETE /identity/v3/projects/e711f1f9a75d4f95ab8906071a2a5196 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:40.943228 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6552bc-2e82-4eb1-ac81-1bb8fb1f334d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:40.943707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6552bc-2e82-4eb1-ac81-1bb8fb1f334d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:40.943800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6552bc-2e82-4eb1-ac81-1bb8fb1f334d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:40.962896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-9b6552bc-2e82-4eb1-ac81-1bb8fb1f334d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:40.964083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-9b6552bc-2e82-4eb1-ac81-1bb8fb1f334d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:41.024479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1155/2306] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:41.034637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540d08-52ac-42e6-b4c3-a484ed22dd5a None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:41.035006 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540d08-52ac-42e6-b4c3-a484ed22dd5a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:41.035351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540d08-52ac-42e6-b4c3-a484ed22dd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:41.056686 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0d540d08-52ac-42e6-b4c3-a484ed22dd5a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:41.057908 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0d540d08-52ac-42e6-b4c3-a484ed22dd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:41.066065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1152/2307] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:41.070465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6552bc-2e82-4eb1-ac81-1bb8fb1f334d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:41.080028 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6552bc-2e82-4eb1-ac81-1bb8fb1f334d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:41.084608 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8bbd3-a824-441f-b070-f899be673063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:41.085203 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8bbd3-a824-441f-b070-f899be673063 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:41.085481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8bbd3-a824-441f-b070-f899be673063 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:41.085754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8bbd3-a824-441f-b070-f899be673063 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:41.086513 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8bbd3-a824-441f-b070-f899be673063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:44:41.086815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8bbd3-a824-441f-b070-f899be673063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:41.088394 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8bbd3-a824-441f-b070-f899be673063 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:41.100363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1156/2308] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:44:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:44:41.105423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d540d08-52ac-42e6-b4c3-a484ed22dd5a None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:41.115197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d540d08-52ac-42e6-b4c3-a484ed22dd5a None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:41.119861 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cab55f-28d0-4059-a016-61ffc3d56de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:41.120739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cab55f-28d0-4059-a016-61ffc3d56de8 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:41.121002 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cab55f-28d0-4059-a016-61ffc3d56de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:41.121279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cab55f-28d0-4059-a016-61ffc3d56de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:41.121941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cab55f-28d0-4059-a016-61ffc3d56de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:41.123894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cab55f-28d0-4059-a016-61ffc3d56de8 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:41.232282 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1153/2309] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:44:41 2022] POST /identity/v3/projects => generated 391 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:44:41.233471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8bbd3-a824-441f-b070-f899be673063 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:41.243799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8bbd3-a824-441f-b070-f899be673063 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:41.319143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c126b-4e07-4eec-b18e-54f360be9eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:41.325646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c126b-4e07-4eec-b18e-54f360be9eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:41.325646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c126b-4e07-4eec-b18e-54f360be9eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:41.325646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c126b-4e07-4eec-b18e-54f360be9eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:41.325646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c126b-4e07-4eec-b18e-54f360be9eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:41.325646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c126b-4e07-4eec-b18e-54f360be9eb9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:41.478408 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1157/2310] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:44:41 2022] POST /identity/v3/users => generated 296 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:44:41.483697 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cab55f-28d0-4059-a016-61ffc3d56de8 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:41.495445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cab55f-28d0-4059-a016-61ffc3d56de8 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:41.500600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6054e0-b75f-4f01-bad2-239659875470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:41.503785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6054e0-b75f-4f01-bad2-239659875470 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:41.503785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6054e0-b75f-4f01-bad2-239659875470 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:41.503785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6054e0-b75f-4f01-bad2-239659875470 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:41.503785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6054e0-b75f-4f01-bad2-239659875470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:41.505212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6054e0-b75f-4f01-bad2-239659875470 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:41.516389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1154/2311] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:41.521304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c126b-4e07-4eec-b18e-54f360be9eb9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:41.532127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c126b-4e07-4eec-b18e-54f360be9eb9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:41.537713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2cabc-db74-4235-8ede-fd821bc04407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:41.538366 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2cabc-db74-4235-8ede-fd821bc04407 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:41.538647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2cabc-db74-4235-8ede-fd821bc04407 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:41.538920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2cabc-db74-4235-8ede-fd821bc04407 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:41.539652 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2cabc-db74-4235-8ede-fd821bc04407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:41.541282 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2cabc-db74-4235-8ede-fd821bc04407 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:41.552271 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1158/2312] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:41.556737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6054e0-b75f-4f01-bad2-239659875470 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:41.567897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6054e0-b75f-4f01-bad2-239659875470 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:41.572371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afa97c-8aea-4fa1-92ad-bea02cb60e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:41.573030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afa97c-8aea-4fa1-92ad-bea02cb60e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:41.573311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afa97c-8aea-4fa1-92ad-bea02cb60e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:41.573609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afa97c-8aea-4fa1-92ad-bea02cb60e2a admin admin] PATH_INFO: `/v3/projects/d4454a5f3e9a4ee399629b082364f03b/users/d47c89c45adc41ac8cfac28c0fe39843/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:41.588408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afa97c-8aea-4fa1-92ad-bea02cb60e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4454a5f3e9a4ee399629b082364f03b, user_id=d47c89c45adc41ac8cfac28c0fe39843, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:41.591226 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afa97c-8aea-4fa1-92ad-bea02cb60e2a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:41.700348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1155/2313] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:44:41 2022] PUT /identity/v3/projects/d4454a5f3e9a4ee399629b082364f03b/users/d47c89c45adc41ac8cfac28c0fe39843/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:41.712423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53012ea8-0e51-4083-b6ef-c9ae9a70d4e4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:41.712871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53012ea8-0e51-4083-b6ef-c9ae9a70d4e4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:41.713154 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53012ea8-0e51-4083-b6ef-c9ae9a70d4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:41.743791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-53012ea8-0e51-4083-b6ef-c9ae9a70d4e4 None None] MFA Rules not processed for user `d47c89c45adc41ac8cfac28c0fe39843`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:41.745239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-53012ea8-0e51-4083-b6ef-c9ae9a70d4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:41.758245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:41.811573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1159/2314] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:41.816458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afa97c-8aea-4fa1-92ad-bea02cb60e2a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:41.878267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afa97c-8aea-4fa1-92ad-bea02cb60e2a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:41.882855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908f01b-bd02-49cd-9810-26cc99539396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:41.883511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908f01b-bd02-49cd-9810-26cc99539396 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:41.883742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908f01b-bd02-49cd-9810-26cc99539396 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:41.884013 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908f01b-bd02-49cd-9810-26cc99539396 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:41.884674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0908f01b-bd02-49cd-9810-26cc99539396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:41.886596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0908f01b-bd02-49cd-9810-26cc99539396 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:41.969733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1156/2315] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:44:41 2022] POST /identity/v3/projects => generated 393 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:44:41.974613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53012ea8-0e51-4083-b6ef-c9ae9a70d4e4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:41.985344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53012ea8-0e51-4083-b6ef-c9ae9a70d4e4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:42.040488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23a5d9-8e07-45d6-83fb-e88b899b651c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:42.041102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23a5d9-8e07-45d6-83fb-e88b899b651c admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:42.041380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23a5d9-8e07-45d6-83fb-e88b899b651c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:42.041655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23a5d9-8e07-45d6-83fb-e88b899b651c admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:42.042348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23a5d9-8e07-45d6-83fb-e88b899b651c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:42.044250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23a5d9-8e07-45d6-83fb-e88b899b651c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:42.180492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1160/2316] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:44:41 2022] POST /identity/v3/users => generated 303 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:44:42.184913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908f01b-bd02-49cd-9810-26cc99539396 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:42.195535 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908f01b-bd02-49cd-9810-26cc99539396 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:42.200306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d419b1-0c9d-43b9-bb78-461d9c0a0582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:42.200916 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d419b1-0c9d-43b9-bb78-461d9c0a0582 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:42.201218 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d419b1-0c9d-43b9-bb78-461d9c0a0582 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:42.201579 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d419b1-0c9d-43b9-bb78-461d9c0a0582 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:42.202302 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d419b1-0c9d-43b9-bb78-461d9c0a0582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:42.204084 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d419b1-0c9d-43b9-bb78-461d9c0a0582 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:42.215057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1157/2317] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:42.219329 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23a5d9-8e07-45d6-83fb-e88b899b651c admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:42.229720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23a5d9-8e07-45d6-83fb-e88b899b651c admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:42.234350 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e976c-761d-4268-9e5b-e5e51d88533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:42.234996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e976c-761d-4268-9e5b-e5e51d88533c admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:42.235274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e976c-761d-4268-9e5b-e5e51d88533c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:42.235547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e976c-761d-4268-9e5b-e5e51d88533c admin admin] PATH_INFO: `/v3/projects/1d8cfc66053041e1a0201739de5597f7/users/c5375b4991564c81a7d84543e8959e84/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:42.249923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e976c-761d-4268-9e5b-e5e51d88533c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d8cfc66053041e1a0201739de5597f7, user_id=c5375b4991564c81a7d84543e8959e84, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:42.252556 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e976c-761d-4268-9e5b-e5e51d88533c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:42.329128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1161/2318] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:44:42 2022] PUT /identity/v3/projects/1d8cfc66053041e1a0201739de5597f7/users/c5375b4991564c81a7d84543e8959e84/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:42.333852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d419b1-0c9d-43b9-bb78-461d9c0a0582 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:42.343786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d419b1-0c9d-43b9-bb78-461d9c0a0582 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:42.376461 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:42.399545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178dacf4-18bc-4286-9e07-363ab1cbeb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:42.400156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dacf4-18bc-4286-9e07-363ab1cbeb13 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:42.400441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dacf4-18bc-4286-9e07-363ab1cbeb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:42.400713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dacf4-18bc-4286-9e07-363ab1cbeb13 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:42.401400 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dacf4-18bc-4286-9e07-363ab1cbeb13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:42.403068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dacf4-18bc-4286-9e07-363ab1cbeb13 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:42.413118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1158/2319] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:42.417523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e976c-761d-4268-9e5b-e5e51d88533c admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:42.428103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e976c-761d-4268-9e5b-e5e51d88533c admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:42.433062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ecd11-fba4-419e-80e4-b4ae92a51388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:42.433668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ecd11-fba4-419e-80e4-b4ae92a51388 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:42.433954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ecd11-fba4-419e-80e4-b4ae92a51388 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:42.434273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ecd11-fba4-419e-80e4-b4ae92a51388 admin admin] PATH_INFO: `/v3/projects/1d8cfc66053041e1a0201739de5597f7/users/c5375b4991564c81a7d84543e8959e84/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:42.436318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ecd11-fba4-419e-80e4-b4ae92a51388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d8cfc66053041e1a0201739de5597f7, user_id=c5375b4991564c81a7d84543e8959e84, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:42.438922 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ecd11-fba4-419e-80e4-b4ae92a51388 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:42.449537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1162/2320] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:44:42 2022] PUT /identity/v3/projects/1d8cfc66053041e1a0201739de5597f7/users/c5375b4991564c81a7d84543e8959e84/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:42.461040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0214f9-a6c8-4864-9431-afc986e7bc39 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:42.461340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0214f9-a6c8-4864-9431-afc986e7bc39 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:42.461625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0214f9-a6c8-4864-9431-afc986e7bc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:42.492001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-3f0214f9-a6c8-4864-9431-afc986e7bc39 None None] MFA Rules not processed for user `c5375b4991564c81a7d84543e8959e84`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:42.493716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3f0214f9-a6c8-4864-9431-afc986e7bc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:42.571014 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1159/2321] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:42 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:42.583886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ecd11-fba4-419e-80e4-b4ae92a51388 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:42.585707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a01ecd11-fba4-419e-80e4-b4ae92a51388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:42.651163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ecd11-fba4-419e-80e4-b4ae92a51388 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:42.655065 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e830e8e-9d78-40a2-b5e2-1d56b8b29ca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:42.655690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e830e8e-9d78-40a2-b5e2-1d56b8b29ca0 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:42.655975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e830e8e-9d78-40a2-b5e2-1d56b8b29ca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:42.656249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e830e8e-9d78-40a2-b5e2-1d56b8b29ca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:42.666212 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e830e8e-9d78-40a2-b5e2-1d56b8b29ca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:42.667842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e830e8e-9d78-40a2-b5e2-1d56b8b29ca0 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:42.678858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1163/2322] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:44:42.726554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0214f9-a6c8-4864-9431-afc986e7bc39 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:42.727952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3f0214f9-a6c8-4864-9431-afc986e7bc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:42.740208 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0214f9-a6c8-4864-9431-afc986e7bc39 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:42.743766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c17cf-696d-4f2b-8845-ccaff045a03c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:42.744369 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c17cf-696d-4f2b-8845-ccaff045a03c service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:42.744649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c17cf-696d-4f2b-8845-ccaff045a03c service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:42.744918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c17cf-696d-4f2b-8845-ccaff045a03c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:42.801939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c17cf-696d-4f2b-8845-ccaff045a03c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:42.803417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c17cf-696d-4f2b-8845-ccaff045a03c service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:42.812961 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1160/2323] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:44:46.239678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d31738-0677-4fe6-b969-a7f56519de16 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:46.240351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d31738-0677-4fe6-b969-a7f56519de16 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:46.240770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d31738-0677-4fe6-b969-a7f56519de16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:46.267046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-a9d31738-0677-4fe6-b969-a7f56519de16 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:46.268891 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a9d31738-0677-4fe6-b969-a7f56519de16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:46.275088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:46.344491 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1164/2324] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:44:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:46.531851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c17cf-696d-4f2b-8845-ccaff045a03c service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:46.533524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-aa8c17cf-696d-4f2b-8845-ccaff045a03c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:46.597205 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:46.611217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c17cf-696d-4f2b-8845-ccaff045a03c service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:46.616860 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74a9e0-ca4f-454e-80aa-61baacbcf5ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:46.631155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74a9e0-ca4f-454e-80aa-61baacbcf5ae service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:46.631155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74a9e0-ca4f-454e-80aa-61baacbcf5ae service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:46.631155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74a9e0-ca4f-454e-80aa-61baacbcf5ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:46.631155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74a9e0-ca4f-454e-80aa-61baacbcf5ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:46.631155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74a9e0-ca4f-454e-80aa-61baacbcf5ae service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:46.636896 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1161/2325] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:44:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:44:49.139738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d31738-0677-4fe6-b969-a7f56519de16 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:49.212571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d31738-0677-4fe6-b969-a7f56519de16 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:49.217607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:49.218284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:49.218563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:49.218836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] PATH_INFO: `/v3/users/d47c89c45adc41ac8cfac28c0fe39843` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:49.219992 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d47c89c45adc41ac8cfac28c0fe39843)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:49.223413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:49.380807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd47c89c45adc41ac8cfac28c0fe39843'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:49.448773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd47c89c45adc41ac8cfac28c0fe39843'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:49.464064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd47c89c45adc41ac8cfac28c0fe39843'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:49.464712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd47c89c45adc41ac8cfac28c0fe39843'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:49.472977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd47c89c45adc41ac8cfac28c0fe39843'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:49.481427 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1165/2326] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:44:49 2022] DELETE /identity/v3/users/d47c89c45adc41ac8cfac28c0fe39843 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:44:49.486854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef5456-a905-4ce1-a02d-5ef2cdbe5c20 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:49.486854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef5456-a905-4ce1-a02d-5ef2cdbe5c20 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:49.486854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef5456-a905-4ce1-a02d-5ef2cdbe5c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:49.508212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-45ef5456-a905-4ce1-a02d-5ef2cdbe5c20 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:49.509465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-45ef5456-a905-4ce1-a02d-5ef2cdbe5c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:49.579562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1162/2327] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:49.590360 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:49.591907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:49.654414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:49.700639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f58402-5ec6-4f80-a679-14a75a310813 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:49.705120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f8c94-68b6-4268-9505-661e68d06e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:49.705383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f8c94-68b6-4268-9505-661e68d06e66 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:49.706211 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f8c94-68b6-4268-9505-661e68d06e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:49.706594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f8c94-68b6-4268-9505-661e68d06e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:49.724659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f8c94-68b6-4268-9505-661e68d06e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:49.727586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f8c94-68b6-4268-9505-661e68d06e66 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:49.741674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1166/2328] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:44:50.255764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef5456-a905-4ce1-a02d-5ef2cdbe5c20 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:50.257645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-45ef5456-a905-4ce1-a02d-5ef2cdbe5c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:50.288363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef5456-a905-4ce1-a02d-5ef2cdbe5c20 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:50.310762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:50.311446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:50.311846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:50.312314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] PATH_INFO: `/v3/projects/d4454a5f3e9a4ee399629b082364f03b` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:50.318371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4454a5f3e9a4ee399629b082364f03b)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:50.321113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:50.492337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4454a5f3e9a4ee399629b082364f03b', 'request_id': 'req-e5d97af0-afd7-4bac-92fd-384620d70ccb'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:50.517239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4454a5f3e9a4ee399629b082364f03b', 'request_id': 'req-e5d97af0-afd7-4bac-92fd-384620d70ccb'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:50.771038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4454a5f3e9a4ee399629b082364f03b', 'request_id': 'req-e5d97af0-afd7-4bac-92fd-384620d70ccb'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:50.783490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] The token cache is being invalidate because project d4454a5f3e9a4ee399629b082364f03b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:44:50.784284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:50.784929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:50.786509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1163/2329] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:44:50 2022] DELETE /identity/v3/projects/d4454a5f3e9a4ee399629b082364f03b => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:50.790833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f8c94-68b6-4268-9505-661e68d06e66 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:50.801571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f8c94-68b6-4268-9505-661e68d06e66 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:50.859719 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf27bf9-bf47-42e6-91d9-19fcabcd2659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:50.860550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf27bf9-bf47-42e6-91d9-19fcabcd2659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:50.860923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf27bf9-bf47-42e6-91d9-19fcabcd2659 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:50.861300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf27bf9-bf47-42e6-91d9-19fcabcd2659 admin admin] PATH_INFO: `/v3/users/c5375b4991564c81a7d84543e8959e84` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:50.863531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf27bf9-bf47-42e6-91d9-19fcabcd2659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5375b4991564c81a7d84543e8959e84)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:50.865763 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf27bf9-bf47-42e6-91d9-19fcabcd2659 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:51.030413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6bf27bf9-bf47-42e6-91d9-19fcabcd2659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5375b4991564c81a7d84543e8959e84'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:51.087651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6bf27bf9-bf47-42e6-91d9-19fcabcd2659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5375b4991564c81a7d84543e8959e84'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:51.127901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6bf27bf9-bf47-42e6-91d9-19fcabcd2659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5375b4991564c81a7d84543e8959e84'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:51.128724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6bf27bf9-bf47-42e6-91d9-19fcabcd2659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5375b4991564c81a7d84543e8959e84'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:51.139745 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6bf27bf9-bf47-42e6-91d9-19fcabcd2659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5375b4991564c81a7d84543e8959e84'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:51.149600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1167/2330] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:44:50 2022] DELETE /identity/v3/users/c5375b4991564c81a7d84543e8959e84 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:51.623023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:51.625101 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:51.643146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d97af0-afd7-4bac-92fd-384620d70ccb admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:51.722574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:51.723255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:51.723537 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:51.723812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] PATH_INFO: `/v3/projects/1d8cfc66053041e1a0201739de5597f7` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:51.724991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d8cfc66053041e1a0201739de5597f7)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:51.726951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:51.740041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:51.834946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d8cfc66053041e1a0201739de5597f7', 'request_id': 'req-bdd5110f-9f08-4284-8cab-a8787e657053'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:51.842617 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d8cfc66053041e1a0201739de5597f7', 'request_id': 'req-bdd5110f-9f08-4284-8cab-a8787e657053'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:51.919290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d8cfc66053041e1a0201739de5597f7', 'request_id': 'req-bdd5110f-9f08-4284-8cab-a8787e657053'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:51.933606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] The token cache is being invalidate because project 1d8cfc66053041e1a0201739de5597f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:44:51.933920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:51.934895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-bdd5110f-9f08-4284-8cab-a8787e657053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:51.940145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1164/2331] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:44:51 2022] DELETE /identity/v3/projects/1d8cfc66053041e1a0201739de5597f7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:51.942094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b2865-ae17-44ca-9987-9553c4d3127e None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:51.943005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b2865-ae17-44ca-9987-9553c4d3127e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:51.943481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b2865-ae17-44ca-9987-9553c4d3127e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:51.965106 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-5d3b2865-ae17-44ca-9987-9553c4d3127e None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:51.966367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5d3b2865-ae17-44ca-9987-9553c4d3127e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:52.028410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:52.038978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1168/2332] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:44:52.048217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad54861-d549-4a5a-8c4f-7e23b0a7bfdd None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:52.048643 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad54861-d549-4a5a-8c4f-7e23b0a7bfdd None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:52.049100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad54861-d549-4a5a-8c4f-7e23b0a7bfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:52.075889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-cad54861-d549-4a5a-8c4f-7e23b0a7bfdd None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:52.077285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-cad54861-d549-4a5a-8c4f-7e23b0a7bfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:52.087393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1165/2333] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:52.091737 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b2865-ae17-44ca-9987-9553c4d3127e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:52.102991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b2865-ae17-44ca-9987-9553c4d3127e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:52.107718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0b895-223d-44f1-bb5b-d51878b7092b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:52.111099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0b895-223d-44f1-bb5b-d51878b7092b admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:52.111099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0b895-223d-44f1-bb5b-d51878b7092b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:52.111099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0b895-223d-44f1-bb5b-d51878b7092b admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:52.111099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0b895-223d-44f1-bb5b-d51878b7092b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:44:52.111099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0b895-223d-44f1-bb5b-d51878b7092b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:52.111735 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0b895-223d-44f1-bb5b-d51878b7092b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:52.126392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1169/2334] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:44:52.130604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad54861-d549-4a5a-8c4f-7e23b0a7bfdd None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:52.141211 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad54861-d549-4a5a-8c4f-7e23b0a7bfdd None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:52.146601 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be973d-c354-4d5d-8321-da5c1e9f42bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:52.147213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be973d-c354-4d5d-8321-da5c1e9f42bb admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:52.147501 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be973d-c354-4d5d-8321-da5c1e9f42bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:52.147767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be973d-c354-4d5d-8321-da5c1e9f42bb admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:52.148448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be973d-c354-4d5d-8321-da5c1e9f42bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:52.150368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be973d-c354-4d5d-8321-da5c1e9f42bb admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:52.288435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1166/2335] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:44:52 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:44:52.289328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0b895-223d-44f1-bb5b-d51878b7092b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:52.301346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0b895-223d-44f1-bb5b-d51878b7092b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:52.358120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d592da-3c7a-4a39-a9a8-a5dd6e0854d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:52.358733 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d592da-3c7a-4a39-a9a8-a5dd6e0854d6 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:52.359016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d592da-3c7a-4a39-a9a8-a5dd6e0854d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:52.359285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d592da-3c7a-4a39-a9a8-a5dd6e0854d6 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:52.359958 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d592da-3c7a-4a39-a9a8-a5dd6e0854d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:52.361983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d592da-3c7a-4a39-a9a8-a5dd6e0854d6 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:52.483094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1170/2336] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:44:52 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:44:52.488017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be973d-c354-4d5d-8321-da5c1e9f42bb admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:52.499433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be973d-c354-4d5d-8321-da5c1e9f42bb admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:52.505216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75b7d2-f12f-4342-9c9b-834abfdcb2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:52.505833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75b7d2-f12f-4342-9c9b-834abfdcb2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:52.506153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75b7d2-f12f-4342-9c9b-834abfdcb2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:52.506434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75b7d2-f12f-4342-9c9b-834abfdcb2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:52.507131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75b7d2-f12f-4342-9c9b-834abfdcb2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:52.508942 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75b7d2-f12f-4342-9c9b-834abfdcb2a7 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:52.522340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1167/2337] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:52.526755 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d592da-3c7a-4a39-a9a8-a5dd6e0854d6 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:52.538284 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d592da-3c7a-4a39-a9a8-a5dd6e0854d6 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:52.543106 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69bd27-3894-43fb-932e-e37827e15f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:52.543716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69bd27-3894-43fb-932e-e37827e15f2f admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:52.543993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69bd27-3894-43fb-932e-e37827e15f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:52.544265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69bd27-3894-43fb-932e-e37827e15f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:52.544947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69bd27-3894-43fb-932e-e37827e15f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:52.546550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69bd27-3894-43fb-932e-e37827e15f2f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:52.557544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1171/2338] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:52.561655 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75b7d2-f12f-4342-9c9b-834abfdcb2a7 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:52.571683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75b7d2-f12f-4342-9c9b-834abfdcb2a7 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:52.576296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730de6b-b071-4ef7-8d88-00b255667a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:52.576948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730de6b-b071-4ef7-8d88-00b255667a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:52.577231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730de6b-b071-4ef7-8d88-00b255667a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:52.577504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730de6b-b071-4ef7-8d88-00b255667a1c admin admin] PATH_INFO: `/v3/projects/4d6ee226ce864eff928214ca9f9d0d95/users/634b68f2431e45c38b4059ff3279eccb/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:52.593122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730de6b-b071-4ef7-8d88-00b255667a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6ee226ce864eff928214ca9f9d0d95, user_id=634b68f2431e45c38b4059ff3279eccb, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:52.595814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730de6b-b071-4ef7-8d88-00b255667a1c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:52.683568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1168/2339] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:44:52 2022] PUT /identity/v3/projects/4d6ee226ce864eff928214ca9f9d0d95/users/634b68f2431e45c38b4059ff3279eccb/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:52.694226 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c24344-22ab-4e7c-a360-489f4fd4ef9d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:52.694510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c24344-22ab-4e7c-a360-489f4fd4ef9d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:52.694795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c24344-22ab-4e7c-a360-489f4fd4ef9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:52.724229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-73c24344-22ab-4e7c-a360-489f4fd4ef9d None None] MFA Rules not processed for user `634b68f2431e45c38b4059ff3279eccb`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:52.725562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-73c24344-22ab-4e7c-a360-489f4fd4ef9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:52.798050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1172/2340] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:52.802906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730de6b-b071-4ef7-8d88-00b255667a1c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:52.870874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730de6b-b071-4ef7-8d88-00b255667a1c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:52.875589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21deee43-f711-402d-a666-28ad289feb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:52.876207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deee43-f711-402d-a666-28ad289feb61 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:52.876492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deee43-f711-402d-a666-28ad289feb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:52.876773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deee43-f711-402d-a666-28ad289feb61 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:52.878147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21deee43-f711-402d-a666-28ad289feb61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:52.883450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21deee43-f711-402d-a666-28ad289feb61 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:52.970566 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1169/2341] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:44:52 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:44:52.974913 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c24344-22ab-4e7c-a360-489f4fd4ef9d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:52.986581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c24344-22ab-4e7c-a360-489f4fd4ef9d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:53.040080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:53.050449 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd65e4-0f6f-45da-b2f8-ecc8b826b24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:53.051009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd65e4-0f6f-45da-b2f8-ecc8b826b24d admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:53.051288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd65e4-0f6f-45da-b2f8-ecc8b826b24d admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:53.051666 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd65e4-0f6f-45da-b2f8-ecc8b826b24d admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:53.052892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd65e4-0f6f-45da-b2f8-ecc8b826b24d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:53.054833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd65e4-0f6f-45da-b2f8-ecc8b826b24d admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:53.142511 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1173/2342] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:44:52 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:44:53.146778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21deee43-f711-402d-a666-28ad289feb61 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:53.157684 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21deee43-f711-402d-a666-28ad289feb61 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:53.163384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87885551-c842-4beb-9b8f-999a35f92488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:53.164008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87885551-c842-4beb-9b8f-999a35f92488 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:53.164292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87885551-c842-4beb-9b8f-999a35f92488 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:53.164567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87885551-c842-4beb-9b8f-999a35f92488 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:53.165326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87885551-c842-4beb-9b8f-999a35f92488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:53.167091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87885551-c842-4beb-9b8f-999a35f92488 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:53.182673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1170/2343] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:53.183701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd65e4-0f6f-45da-b2f8-ecc8b826b24d admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:53.196402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd65e4-0f6f-45da-b2f8-ecc8b826b24d admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:53.201323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d4254-f264-4b41-af9d-5616732091ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:53.202000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d4254-f264-4b41-af9d-5616732091ee admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:53.202317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d4254-f264-4b41-af9d-5616732091ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:53.202596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d4254-f264-4b41-af9d-5616732091ee admin admin] PATH_INFO: `/v3/projects/4cefb2b23d5443d7877e7cff5c2a2cfc/users/27e77748eae84415ad1cb02a3a95756f/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:53.218894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d4254-f264-4b41-af9d-5616732091ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cefb2b23d5443d7877e7cff5c2a2cfc, user_id=27e77748eae84415ad1cb02a3a95756f, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:53.222539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d4254-f264-4b41-af9d-5616732091ee admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:53.307679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1174/2344] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:44:53 2022] PUT /identity/v3/projects/4cefb2b23d5443d7877e7cff5c2a2cfc/users/27e77748eae84415ad1cb02a3a95756f/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:44:53.311847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87885551-c842-4beb-9b8f-999a35f92488 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:53.321944 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87885551-c842-4beb-9b8f-999a35f92488 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:53.331883 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:53.399256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03be0a-d9dc-490f-8923-e63e77421ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:53.399986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03be0a-d9dc-490f-8923-e63e77421ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:53.400236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03be0a-d9dc-490f-8923-e63e77421ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:53.400545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03be0a-d9dc-490f-8923-e63e77421ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:53.401216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03be0a-d9dc-490f-8923-e63e77421ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:53.407042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03be0a-d9dc-490f-8923-e63e77421ba4 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:53.421545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1171/2345] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:44:53.426036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d4254-f264-4b41-af9d-5616732091ee admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:53.436991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d4254-f264-4b41-af9d-5616732091ee admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:53.442307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38240bec-6e87-4fcf-9c18-e4794b1f0450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:53.443121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240bec-6e87-4fcf-9c18-e4794b1f0450 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:53.443493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240bec-6e87-4fcf-9c18-e4794b1f0450 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:53.443884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240bec-6e87-4fcf-9c18-e4794b1f0450 admin admin] PATH_INFO: `/v3/projects/4cefb2b23d5443d7877e7cff5c2a2cfc/users/27e77748eae84415ad1cb02a3a95756f/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:53.446099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38240bec-6e87-4fcf-9c18-e4794b1f0450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cefb2b23d5443d7877e7cff5c2a2cfc, user_id=27e77748eae84415ad1cb02a3a95756f, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:53.450171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38240bec-6e87-4fcf-9c18-e4794b1f0450 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:53.463575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1175/2346] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:44:53 2022] PUT /identity/v3/projects/4cefb2b23d5443d7877e7cff5c2a2cfc/users/27e77748eae84415ad1cb02a3a95756f/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:53.475442 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b4b18-5a5b-4537-af74-5a2e4752d666 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:53.476253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b4b18-5a5b-4537-af74-5a2e4752d666 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:53.476735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b4b18-5a5b-4537-af74-5a2e4752d666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:53.512236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-b82b4b18-5a5b-4537-af74-5a2e4752d666 None None] MFA Rules not processed for user `27e77748eae84415ad1cb02a3a95756f`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:53.513591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b82b4b18-5a5b-4537-af74-5a2e4752d666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:53.615099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1172/2347] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:53.623497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38240bec-6e87-4fcf-9c18-e4794b1f0450 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:53.625469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-38240bec-6e87-4fcf-9c18-e4794b1f0450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:53.698689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38240bec-6e87-4fcf-9c18-e4794b1f0450 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:53.702434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29419b-63df-4d99-9ada-ff32cda36202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:53.703046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29419b-63df-4d99-9ada-ff32cda36202 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:53.703324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29419b-63df-4d99-9ada-ff32cda36202 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:53.703593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29419b-63df-4d99-9ada-ff32cda36202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:53.712658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29419b-63df-4d99-9ada-ff32cda36202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:53.714114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29419b-63df-4d99-9ada-ff32cda36202 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:53.724140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1176/2348] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:44:54.301685 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b4b18-5a5b-4537-af74-5a2e4752d666 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:54.378331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b4b18-5a5b-4537-af74-5a2e4752d666 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:54.383433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:54.384106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:54.384396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:54.384658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] PATH_INFO: `/v3/users/634b68f2431e45c38b4059ff3279eccb` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:54.385827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=634b68f2431e45c38b4059ff3279eccb)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:54.388036 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:54.552336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:54.565663 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '634b68f2431e45c38b4059ff3279eccb'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:54.616368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '634b68f2431e45c38b4059ff3279eccb'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:54.617363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '634b68f2431e45c38b4059ff3279eccb'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:54.650792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '634b68f2431e45c38b4059ff3279eccb'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:54.668290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '634b68f2431e45c38b4059ff3279eccb'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:54.682336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1173/2349] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:44:54 2022] DELETE /identity/v3/users/634b68f2431e45c38b4059ff3279eccb => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:54.685225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968a880-8c72-4b37-990a-cc621762f67f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:54.685683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968a880-8c72-4b37-990a-cc621762f67f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:54.685797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968a880-8c72-4b37-990a-cc621762f67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:54.707579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-e968a880-8c72-4b37-990a-cc621762f67f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:54.708793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e968a880-8c72-4b37-990a-cc621762f67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:54.790770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1177/2350] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:54.797163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:54.798765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:54.869201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6163b9d-77bd-465f-b925-bdb6ea6dcd9e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:54.873531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d5cc6-9211-4be3-a72b-13192702f8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:54.874190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d5cc6-9211-4be3-a72b-13192702f8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:54.874473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d5cc6-9211-4be3-a72b-13192702f8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:54.874747 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d5cc6-9211-4be3-a72b-13192702f8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:54.884731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d5cc6-9211-4be3-a72b-13192702f8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:54.886245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d5cc6-9211-4be3-a72b-13192702f8d0 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:54.896499 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1174/2351] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:44:55.325127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968a880-8c72-4b37-990a-cc621762f67f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:55.325127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e968a880-8c72-4b37-990a-cc621762f67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:55.342377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968a880-8c72-4b37-990a-cc621762f67f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:55.347802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:55.348471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:55.348882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:55.349113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] PATH_INFO: `/v3/projects/4d6ee226ce864eff928214ca9f9d0d95` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:55.350687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d6ee226ce864eff928214ca9f9d0d95)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:55.352623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:55.473784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:55.478337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d6ee226ce864eff928214ca9f9d0d95', 'request_id': 'req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:55.934326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d6ee226ce864eff928214ca9f9d0d95', 'request_id': 'req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:55.940417 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d6ee226ce864eff928214ca9f9d0d95', 'request_id': 'req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:55.951846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] The token cache is being invalidate because project 4d6ee226ce864eff928214ca9f9d0d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:44:55.973888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:55.973888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:55.973888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1178/2352] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:44:55 2022] DELETE /identity/v3/projects/4d6ee226ce864eff928214ca9f9d0d95 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:55.976752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d5cc6-9211-4be3-a72b-13192702f8d0 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:55.986594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d5cc6-9211-4be3-a72b-13192702f8d0 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:56.052082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84a233-2b2d-4c6c-b413-705afccdf525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:56.052082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a233-2b2d-4c6c-b413-705afccdf525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:56.052082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a233-2b2d-4c6c-b413-705afccdf525 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:56.052082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a233-2b2d-4c6c-b413-705afccdf525 admin admin] PATH_INFO: `/v3/users/27e77748eae84415ad1cb02a3a95756f` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:56.052510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84a233-2b2d-4c6c-b413-705afccdf525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e77748eae84415ad1cb02a3a95756f)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:56.052510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84a233-2b2d-4c6c-b413-705afccdf525 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:56.242093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-de84a233-2b2d-4c6c-b413-705afccdf525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e77748eae84415ad1cb02a3a95756f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:56.299424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-de84a233-2b2d-4c6c-b413-705afccdf525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e77748eae84415ad1cb02a3a95756f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:56.300131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-de84a233-2b2d-4c6c-b413-705afccdf525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e77748eae84415ad1cb02a3a95756f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:56.302778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:56.315192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-de84a233-2b2d-4c6c-b413-705afccdf525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e77748eae84415ad1cb02a3a95756f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:56.324784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-de84a233-2b2d-4c6c-b413-705afccdf525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e77748eae84415ad1cb02a3a95756f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:56.332169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1175/2353] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:44:55 2022] DELETE /identity/v3/users/27e77748eae84415ad1cb02a3a95756f => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:44:56.851114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:56.852677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:56.881040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da85fc9-8a9b-4d68-ada8-fd628ebc3c6f admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:56.987542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:56.988417 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:56.989571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:56.989888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] PATH_INFO: `/v3/projects/4cefb2b23d5443d7877e7cff5c2a2cfc` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:56.992476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cefb2b23d5443d7877e7cff5c2a2cfc)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:56.994184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:57.139894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cefb2b23d5443d7877e7cff5c2a2cfc', 'request_id': 'req-2352703a-e957-462c-9291-5f69285d4a0b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:57.226449 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cefb2b23d5443d7877e7cff5c2a2cfc', 'request_id': 'req-2352703a-e957-462c-9291-5f69285d4a0b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:57.235189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cefb2b23d5443d7877e7cff5c2a2cfc', 'request_id': 'req-2352703a-e957-462c-9291-5f69285d4a0b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:57.254434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] The token cache is being invalidate because project 4cefb2b23d5443d7877e7cff5c2a2cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:44:57.254781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:57.262486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2352703a-e957-462c-9291-5f69285d4a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:44:57.268153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1179/2354] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:44:56 2022] DELETE /identity/v3/projects/4cefb2b23d5443d7877e7cff5c2a2cfc => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:57.269532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f19ef-4138-4d58-a12d-90655ca9c86c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:57.271682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f19ef-4138-4d58-a12d-90655ca9c86c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:57.271682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f19ef-4138-4d58-a12d-90655ca9c86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:57.305268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-b30f19ef-4138-4d58-a12d-90655ca9c86c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:57.306638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b30f19ef-4138-4d58-a12d-90655ca9c86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:57.385663 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1176/2355] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:44:57.394900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2735176-0040-42ac-9785-37fe49e7bf15 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:57.395370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2735176-0040-42ac-9785-37fe49e7bf15 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:57.395991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2735176-0040-42ac-9785-37fe49e7bf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:57.440869 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c2735176-0040-42ac-9785-37fe49e7bf15 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:57.443399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c2735176-0040-42ac-9785-37fe49e7bf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:57.452026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1180/2356] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:44:57.456498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f19ef-4138-4d58-a12d-90655ca9c86c None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:57.469498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f19ef-4138-4d58-a12d-90655ca9c86c None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:57.475813 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59415ec6-78ed-477e-a2fa-175b1b91ad9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:57.476491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59415ec6-78ed-477e-a2fa-175b1b91ad9d admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:57.476774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59415ec6-78ed-477e-a2fa-175b1b91ad9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:57.477065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59415ec6-78ed-477e-a2fa-175b1b91ad9d admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:57.477772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59415ec6-78ed-477e-a2fa-175b1b91ad9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:44:57.478115 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59415ec6-78ed-477e-a2fa-175b1b91ad9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:57.479760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59415ec6-78ed-477e-a2fa-175b1b91ad9d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:57.494437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1177/2357] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:44:57.499426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2735176-0040-42ac-9785-37fe49e7bf15 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:57.529279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2735176-0040-42ac-9785-37fe49e7bf15 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:57.554080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749966e4-9ecc-4d49-8116-1c1e6cab59d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:57.554080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749966e4-9ecc-4d49-8116-1c1e6cab59d4 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:57.554080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749966e4-9ecc-4d49-8116-1c1e6cab59d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:57.554080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749966e4-9ecc-4d49-8116-1c1e6cab59d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:57.554552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749966e4-9ecc-4d49-8116-1c1e6cab59d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:57.554552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749966e4-9ecc-4d49-8116-1c1e6cab59d4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:57.646837 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1181/2358] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:44:57 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:44:57.650786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59415ec6-78ed-477e-a2fa-175b1b91ad9d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:57.666243 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59415ec6-78ed-477e-a2fa-175b1b91ad9d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:57.742803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd4368-cf24-4614-90a6-77afba0e3cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:57.743540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd4368-cf24-4614-90a6-77afba0e3cdc admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:57.743896 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd4368-cf24-4614-90a6-77afba0e3cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:57.744268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd4368-cf24-4614-90a6-77afba0e3cdc admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:57.745027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd4368-cf24-4614-90a6-77afba0e3cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:57.747448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd4368-cf24-4614-90a6-77afba0e3cdc admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:57.833747 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1178/2359] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:44:57 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:44:57.838329 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749966e4-9ecc-4d49-8116-1c1e6cab59d4 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:57.851859 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749966e4-9ecc-4d49-8116-1c1e6cab59d4 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:57.857534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa77fe-a913-4d95-b491-3f9b7ead6b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:57.858194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa77fe-a913-4d95-b491-3f9b7ead6b34 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:57.858503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa77fe-a913-4d95-b491-3f9b7ead6b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:57.858860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa77fe-a913-4d95-b491-3f9b7ead6b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:57.862560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa77fe-a913-4d95-b491-3f9b7ead6b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:57.862560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa77fe-a913-4d95-b491-3f9b7ead6b34 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:57.878232 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1182/2360] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:44:57.879622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd4368-cf24-4614-90a6-77afba0e3cdc admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:57.892180 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd4368-cf24-4614-90a6-77afba0e3cdc admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:57.898815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f3bb4-c6fe-4a75-84bb-0703aae9d5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:57.899249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f3bb4-c6fe-4a75-84bb-0703aae9d5db admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:57.899531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f3bb4-c6fe-4a75-84bb-0703aae9d5db admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:57.899823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f3bb4-c6fe-4a75-84bb-0703aae9d5db admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:57.900557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f3bb4-c6fe-4a75-84bb-0703aae9d5db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:57.902224 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f3bb4-c6fe-4a75-84bb-0703aae9d5db admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:57.918202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1179/2361] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:44:57.922467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa77fe-a913-4d95-b491-3f9b7ead6b34 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:57.939085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa77fe-a913-4d95-b491-3f9b7ead6b34 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:57.945003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd268a8-5080-401c-8cdc-a4df09bd708b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:57.945701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd268a8-5080-401c-8cdc-a4df09bd708b admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:57.946132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd268a8-5080-401c-8cdc-a4df09bd708b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:57.946409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd268a8-5080-401c-8cdc-a4df09bd708b admin admin] PATH_INFO: `/v3/projects/962199458c7a4ae8850039663b00572c/users/8675f93a96de42c9802fc38705c50e0c/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:57.967963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd268a8-5080-401c-8cdc-a4df09bd708b admin admin] RBAC: Authorizing `identity:create_grant(project_id=962199458c7a4ae8850039663b00572c, user_id=8675f93a96de42c9802fc38705c50e0c, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:57.970844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd268a8-5080-401c-8cdc-a4df09bd708b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:58.088979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:58.097744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1183/2362] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:44:57 2022] PUT /identity/v3/projects/962199458c7a4ae8850039663b00572c/users/8675f93a96de42c9802fc38705c50e0c/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:44:58.108829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275927bb-c79c-4698-b1c0-e090a27054fe None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:58.109019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275927bb-c79c-4698-b1c0-e090a27054fe None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:58.112333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275927bb-c79c-4698-b1c0-e090a27054fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:58.146166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-275927bb-c79c-4698-b1c0-e090a27054fe None None] MFA Rules not processed for user `8675f93a96de42c9802fc38705c50e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:44:58.147632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-275927bb-c79c-4698-b1c0-e090a27054fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:58.180633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:44:58.220729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1180/2363] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:44:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:44:58.231663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd268a8-5080-401c-8cdc-a4df09bd708b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:44:58.233450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8fd268a8-5080-401c-8cdc-a4df09bd708b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:44:58.292315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd268a8-5080-401c-8cdc-a4df09bd708b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:44:58.301824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0f3e9-3f5d-4ae2-abbe-b7b2e43313e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:44:58.304429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0f3e9-3f5d-4ae2-abbe-b7b2e43313e5 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:44:58.304429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0f3e9-3f5d-4ae2-abbe-b7b2e43313e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:44:58.304429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0f3e9-3f5d-4ae2-abbe-b7b2e43313e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:44:58.314229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0f3e9-3f5d-4ae2-abbe-b7b2e43313e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:44:58.315863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0f3e9-3f5d-4ae2-abbe-b7b2e43313e5 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:44:58.329389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1184/2364] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:44:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:07.920717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275927bb-c79c-4698-b1c0-e090a27054fe None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:07.926768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-275927bb-c79c-4698-b1c0-e090a27054fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:07.991241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275927bb-c79c-4698-b1c0-e090a27054fe None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:07.996384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:07.997028 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:07.997312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:07.997594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] PATH_INFO: `/v3/users/df8864b104d7435882590db91b7e9f7f` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:07.998781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df8864b104d7435882590db91b7e9f7f)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:08.000702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:08.147595 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df8864b104d7435882590db91b7e9f7f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:08.197080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df8864b104d7435882590db91b7e9f7f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:08.197745 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df8864b104d7435882590db91b7e9f7f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:08.210211 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df8864b104d7435882590db91b7e9f7f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:08.217691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df8864b104d7435882590db91b7e9f7f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:08.224048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1181/2365] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:45:07 2022] DELETE /identity/v3/users/df8864b104d7435882590db91b7e9f7f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:08.229244 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738edd92-e9e6-40dc-b2de-caca4bc2906d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:08.229763 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738edd92-e9e6-40dc-b2de-caca4bc2906d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:08.229881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738edd92-e9e6-40dc-b2de-caca4bc2906d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:08.248848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-738edd92-e9e6-40dc-b2de-caca4bc2906d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:08.250127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-738edd92-e9e6-40dc-b2de-caca4bc2906d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:08.308783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1185/2366] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:08.318644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:08.320085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:08.375776 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:08.380008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7867ef-bd4f-439c-9d0c-3f8ec11a0a51 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:08.384005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eec5d6-2aa8-4b96-b2ff-dd5757f61ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:08.384621 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eec5d6-2aa8-4b96-b2ff-dd5757f61ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:08.384907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eec5d6-2aa8-4b96-b2ff-dd5757f61ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:08.385179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eec5d6-2aa8-4b96-b2ff-dd5757f61ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:08.394340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eec5d6-2aa8-4b96-b2ff-dd5757f61ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:08.395770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eec5d6-2aa8-4b96-b2ff-dd5757f61ce2 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:08.405882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1182/2367] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:08.775549 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738edd92-e9e6-40dc-b2de-caca4bc2906d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:08.776993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-738edd92-e9e6-40dc-b2de-caca4bc2906d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:08.789138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738edd92-e9e6-40dc-b2de-caca4bc2906d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:08.794330 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:08.795030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:08.795315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:08.795585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] PATH_INFO: `/v3/projects/a8170a9eef56402289850f880fe0dc44` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:08.796739 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8170a9eef56402289850f880fe0dc44)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:08.799706 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:08.942098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8170a9eef56402289850f880fe0dc44', 'request_id': 'req-4c103559-902d-4f01-9786-2aac28593a7a'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:09.009017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8170a9eef56402289850f880fe0dc44', 'request_id': 'req-4c103559-902d-4f01-9786-2aac28593a7a'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:09.016237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8170a9eef56402289850f880fe0dc44', 'request_id': 'req-4c103559-902d-4f01-9786-2aac28593a7a'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:09.031518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] The token cache is being invalidate because project a8170a9eef56402289850f880fe0dc44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:45:09.031919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:09.032838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-4c103559-902d-4f01-9786-2aac28593a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:09.050578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1186/2368] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:45:08 2022] DELETE /identity/v3/projects/a8170a9eef56402289850f880fe0dc44 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:10.304561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eec5d6-2aa8-4b96-b2ff-dd5757f61ce2 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:10.306070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-25eec5d6-2aa8-4b96-b2ff-dd5757f61ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:10.324046 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eec5d6-2aa8-4b96-b2ff-dd5757f61ce2 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:10.380380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4db798-d206-404f-b900-a40e74ea7029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:10.380939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4db798-d206-404f-b900-a40e74ea7029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:10.381224 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4db798-d206-404f-b900-a40e74ea7029 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:10.381551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4db798-d206-404f-b900-a40e74ea7029 admin admin] PATH_INFO: `/v3/users/8675f93a96de42c9802fc38705c50e0c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:10.382747 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4db798-d206-404f-b900-a40e74ea7029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8675f93a96de42c9802fc38705c50e0c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:10.384688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4db798-d206-404f-b900-a40e74ea7029 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:10.589292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7f4db798-d206-404f-b900-a40e74ea7029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8675f93a96de42c9802fc38705c50e0c'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:10.616890 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c1692-221c-48b8-b8c7-8d131d0c5d79 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:10.617465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c1692-221c-48b8-b8c7-8d131d0c5d79 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:10.617465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c1692-221c-48b8-b8c7-8d131d0c5d79 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:10.618970 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1187/2369] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:45:10.625831 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068035-6435-449f-9f1b-80e3cb47ebe9 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:10.626201 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068035-6435-449f-9f1b-80e3cb47ebe9 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:10.627219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068035-6435-449f-9f1b-80e3cb47ebe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:10.637797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7f4db798-d206-404f-b900-a40e74ea7029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8675f93a96de42c9802fc38705c50e0c'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:10.638394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7f4db798-d206-404f-b900-a40e74ea7029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8675f93a96de42c9802fc38705c50e0c'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:10.650554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-83068035-6435-449f-9f1b-80e3cb47ebe9 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:10.652448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-83068035-6435-449f-9f1b-80e3cb47ebe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:10.654492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7f4db798-d206-404f-b900-a40e74ea7029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8675f93a96de42c9802fc38705c50e0c'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:10.662358 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-7f4db798-d206-404f-b900-a40e74ea7029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8675f93a96de42c9802fc38705c50e0c'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:10.670220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1183/2370] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:45:10 2022] DELETE /identity/v3/users/8675f93a96de42c9802fc38705c50e0c => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:10.674454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd526d6-ee0e-4584-9de6-8aa4a58b249f None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:10.674828 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd526d6-ee0e-4584-9de6-8aa4a58b249f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:10.676091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd526d6-ee0e-4584-9de6-8aa4a58b249f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:10.678720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:10.700246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-bcd526d6-ee0e-4584-9de6-8aa4a58b249f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:10.702351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bcd526d6-ee0e-4584-9de6-8aa4a58b249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:10.750388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1188/2371] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:45:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:45:10.762712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068035-6435-449f-9f1b-80e3cb47ebe9 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:10.764165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-83068035-6435-449f-9f1b-80e3cb47ebe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:10.782099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1184/2372] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:45:10.793780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd526d6-ee0e-4584-9de6-8aa4a58b249f None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:10.795195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bcd526d6-ee0e-4584-9de6-8aa4a58b249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:10.809962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:10.824119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068035-6435-449f-9f1b-80e3cb47ebe9 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:10.855263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd526d6-ee0e-4584-9de6-8aa4a58b249f None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:10.859274 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f8638-c7c8-4381-ba2c-8ce19966b08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:10.859906 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f8638-c7c8-4381-ba2c-8ce19966b08c service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:10.860327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f8638-c7c8-4381-ba2c-8ce19966b08c service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:10.860979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f8638-c7c8-4381-ba2c-8ce19966b08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:10.870623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f8638-c7c8-4381-ba2c-8ce19966b08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:10.873010 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f8638-c7c8-4381-ba2c-8ce19966b08c service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:10.873713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b9dfc-ee08-42ed-96a4-25246353eca6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a9a6bcfa1b469298774e09a8912b93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:10.874370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b9dfc-ee08-42ed-96a4-25246353eca6 service glance] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:10.874651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b9dfc-ee08-42ed-96a4-25246353eca6 service glance] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:10.874922 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b9dfc-ee08-42ed-96a4-25246353eca6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:10.882755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1185/2373] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:45:10.944404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b9dfc-ee08-42ed-96a4-25246353eca6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:10.945839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b9dfc-ee08-42ed-96a4-25246353eca6 service glance] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:10.957447 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1189/2374] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:45:11.172173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3210d03-4e5f-48b0-9e7a-2d029ba0b2df None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:11.172567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3210d03-4e5f-48b0-9e7a-2d029ba0b2df None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:11.173285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3210d03-4e5f-48b0-9e7a-2d029ba0b2df None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:11.175122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1186/2375] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:45:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 21:45:11.180769 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50079af-03a7-4f43-b048-f1d770bb38d4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:11.181405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50079af-03a7-4f43-b048-f1d770bb38d4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:11.181853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50079af-03a7-4f43-b048-f1d770bb38d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:11.203437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f50079af-03a7-4f43-b048-f1d770bb38d4 None None] MFA Rules not processed for user `c98761f81496456fb3b5a5cffd4ce763`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:11.204644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f50079af-03a7-4f43-b048-f1d770bb38d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:11.211553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1190/2376] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:45:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:11.223560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3210d03-4e5f-48b0-9e7a-2d029ba0b2df None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:11.236146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3210d03-4e5f-48b0-9e7a-2d029ba0b2df None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:11.239724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeace63-78dd-4a2a-beb7-7f73a60e8b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:11.240325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeace63-78dd-4a2a-beb7-7f73a60e8b46 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:11.240606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeace63-78dd-4a2a-beb7-7f73a60e8b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:11.241049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeace63-78dd-4a2a-beb7-7f73a60e8b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:11.248900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeace63-78dd-4a2a-beb7-7f73a60e8b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:11.250365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeace63-78dd-4a2a-beb7-7f73a60e8b46 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:11.258074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1187/2377] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:11.342156 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ceb7be-a48a-43eb-b0b1-edf0aa5e4b3f None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:11.342482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ceb7be-a48a-43eb-b0b1-edf0aa5e4b3f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:11.342784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ceb7be-a48a-43eb-b0b1-edf0aa5e4b3f None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:11.344130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1191/2378] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:45:11.349635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99190123-ddc9-4fbf-9165-4758797293e2 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:11.350117 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99190123-ddc9-4fbf-9165-4758797293e2 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:11.350311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99190123-ddc9-4fbf-9165-4758797293e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:11.358281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ceb7be-a48a-43eb-b0b1-edf0aa5e4b3f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:11.361250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d9ceb7be-a48a-43eb-b0b1-edf0aa5e4b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:11.379518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ceb7be-a48a-43eb-b0b1-edf0aa5e4b3f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:11.384414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:11.385057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:11.385453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:11.386042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] PATH_INFO: `/v3/projects/962199458c7a4ae8850039663b00572c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:11.387135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=962199458c7a4ae8850039663b00572c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:11.387232 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-99190123-ddc9-4fbf-9165-4758797293e2 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:11.388492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-99190123-ddc9-4fbf-9165-4758797293e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:11.389172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:11.391092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:11.401192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1188/2379] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:45:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:11.411889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99190123-ddc9-4fbf-9165-4758797293e2 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:11.413456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-99190123-ddc9-4fbf-9165-4758797293e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:11.425606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99190123-ddc9-4fbf-9165-4758797293e2 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:11.429417 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d1263-a458-4737-96c4-d6bb96b26d36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:11.430054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d1263-a458-4737-96c4-d6bb96b26d36 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:11.430337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d1263-a458-4737-96c4-d6bb96b26d36 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:11.430756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d1263-a458-4737-96c4-d6bb96b26d36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:11.441541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d1263-a458-4737-96c4-d6bb96b26d36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:11.442995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d1263-a458-4737-96c4-d6bb96b26d36 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:11.453893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1189/2380] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:45:11.522932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '962199458c7a4ae8850039663b00572c', 'request_id': 'req-7b559497-1c81-4551-b45b-f204c51714d6'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:11.591375 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '962199458c7a4ae8850039663b00572c', 'request_id': 'req-7b559497-1c81-4551-b45b-f204c51714d6'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:11.597632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '962199458c7a4ae8850039663b00572c', 'request_id': 'req-7b559497-1c81-4551-b45b-f204c51714d6'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:11.609623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] The token cache is being invalidate because project 962199458c7a4ae8850039663b00572c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:45:11.609915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:11.610675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b559497-1c81-4551-b45b-f204c51714d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:11.616623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1192/2381] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:45:11 2022] DELETE /identity/v3/projects/962199458c7a4ae8850039663b00572c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:45:11.617394 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538f7b6-198c-4038-8314-f8818b33d7b6 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:11.617689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538f7b6-198c-4038-8314-f8818b33d7b6 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:11.617968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538f7b6-198c-4038-8314-f8818b33d7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:11.638753 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-e538f7b6-198c-4038-8314-f8818b33d7b6 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:11.639975 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e538f7b6-198c-4038-8314-f8818b33d7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:11.700173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1190/2382] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:11.768093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61743c18-cf7c-4638-b964-434c33397896 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:11.768650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61743c18-cf7c-4638-b964-434c33397896 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:11.768650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61743c18-cf7c-4638-b964-434c33397896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:11.790090 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-61743c18-cf7c-4638-b964-434c33397896 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:11.791306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-61743c18-cf7c-4638-b964-434c33397896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:11.800201 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1193/2383] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:11.804650 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538f7b6-198c-4038-8314-f8818b33d7b6 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:11.814810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538f7b6-198c-4038-8314-f8818b33d7b6 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:11.819613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495e44b-7c49-4d17-a6c4-a2fd853bbddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:11.820233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495e44b-7c49-4d17-a6c4-a2fd853bbddf admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:11.820521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495e44b-7c49-4d17-a6c4-a2fd853bbddf admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:11.822085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495e44b-7c49-4d17-a6c4-a2fd853bbddf admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:11.822085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495e44b-7c49-4d17-a6c4-a2fd853bbddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:45:11.822085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495e44b-7c49-4d17-a6c4-a2fd853bbddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:11.823630 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495e44b-7c49-4d17-a6c4-a2fd853bbddf admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:11.836701 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1191/2384] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:45:11.846630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61743c18-cf7c-4638-b964-434c33397896 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:11.858357 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61743c18-cf7c-4638-b964-434c33397896 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:11.864011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79369612-984c-42ef-8b7c-1771ac023905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:11.864703 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79369612-984c-42ef-8b7c-1771ac023905 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:11.865860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79369612-984c-42ef-8b7c-1771ac023905 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:11.865860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79369612-984c-42ef-8b7c-1771ac023905 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:11.866390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79369612-984c-42ef-8b7c-1771ac023905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:11.868359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79369612-984c-42ef-8b7c-1771ac023905 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:11.939461 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1194/2385] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:45:11 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:45:11.940740 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495e44b-7c49-4d17-a6c4-a2fd853bbddf admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:11.951348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495e44b-7c49-4d17-a6c4-a2fd853bbddf admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:12.007285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084a972-56d9-4a96-aad5-69bfc8758f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:12.007895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084a972-56d9-4a96-aad5-69bfc8758f4c admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:12.008178 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084a972-56d9-4a96-aad5-69bfc8758f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:12.008452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084a972-56d9-4a96-aad5-69bfc8758f4c admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:12.009109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084a972-56d9-4a96-aad5-69bfc8758f4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:12.011250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084a972-56d9-4a96-aad5-69bfc8758f4c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:12.140395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:12.146565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1192/2386] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:45:11 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:45:12.150747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79369612-984c-42ef-8b7c-1771ac023905 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:12.161688 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79369612-984c-42ef-8b7c-1771ac023905 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:12.166415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef6dc-60d1-4084-b6f6-c86eabc3b130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:12.167042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef6dc-60d1-4084-b6f6-c86eabc3b130 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:12.167319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef6dc-60d1-4084-b6f6-c86eabc3b130 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:12.167593 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef6dc-60d1-4084-b6f6-c86eabc3b130 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:12.168286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ef6dc-60d1-4084-b6f6-c86eabc3b130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:12.169866 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ef6dc-60d1-4084-b6f6-c86eabc3b130 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:12.180578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1195/2387] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:45:12.184673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084a972-56d9-4a96-aad5-69bfc8758f4c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:12.194930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084a972-56d9-4a96-aad5-69bfc8758f4c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:12.200876 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a72cbf-0d1f-43ee-bd27-ab9347f8960c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:12.201536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a72cbf-0d1f-43ee-bd27-ab9347f8960c admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:12.201756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a72cbf-0d1f-43ee-bd27-ab9347f8960c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:12.202053 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a72cbf-0d1f-43ee-bd27-ab9347f8960c admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:12.202748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a72cbf-0d1f-43ee-bd27-ab9347f8960c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:12.204325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a72cbf-0d1f-43ee-bd27-ab9347f8960c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:12.215421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1193/2388] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:45:12.219496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef6dc-60d1-4084-b6f6-c86eabc3b130 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:12.229596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef6dc-60d1-4084-b6f6-c86eabc3b130 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:12.234326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e58dc-bac1-475d-af12-d859c5c7c1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:12.234986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e58dc-bac1-475d-af12-d859c5c7c1f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:12.235265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e58dc-bac1-475d-af12-d859c5c7c1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:12.235628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e58dc-bac1-475d-af12-d859c5c7c1f9 admin admin] PATH_INFO: `/v3/projects/3820366e057b4c50a74ce1b871575c90/users/276f842586c34e509e5e87058c725177/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:12.250812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e58dc-bac1-475d-af12-d859c5c7c1f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3820366e057b4c50a74ce1b871575c90, user_id=276f842586c34e509e5e87058c725177, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:12.253552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e58dc-bac1-475d-af12-d859c5c7c1f9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:12.323171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1196/2389] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:45:12 2022] PUT /identity/v3/projects/3820366e057b4c50a74ce1b871575c90/users/276f842586c34e509e5e87058c725177/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:12.333542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12805d-8aaf-4341-98f9-25e71427ba3c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:12.333838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12805d-8aaf-4341-98f9-25e71427ba3c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:12.334150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12805d-8aaf-4341-98f9-25e71427ba3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:12.362673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ca12805d-8aaf-4341-98f9-25e71427ba3c None None] MFA Rules not processed for user `276f842586c34e509e5e87058c725177`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:12.363897 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ca12805d-8aaf-4341-98f9-25e71427ba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:12.430617 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1194/2390] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:12.436006 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e58dc-bac1-475d-af12-d859c5c7c1f9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:12.502829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e58dc-bac1-475d-af12-d859c5c7c1f9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:12.507637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5079b81-3a28-43ef-9da9-52f4aed2530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:12.508245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079b81-3a28-43ef-9da9-52f4aed2530e admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:12.508524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079b81-3a28-43ef-9da9-52f4aed2530e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:12.508795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079b81-3a28-43ef-9da9-52f4aed2530e admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:12.509454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5079b81-3a28-43ef-9da9-52f4aed2530e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:12.511664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5079b81-3a28-43ef-9da9-52f4aed2530e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:12.601684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1197/2391] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:45:12 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:45:12.603503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12805d-8aaf-4341-98f9-25e71427ba3c None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:12.616756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12805d-8aaf-4341-98f9-25e71427ba3c None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:12.675591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0d754-9e13-43e4-a1d4-ea27e403f385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:12.676214 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0d754-9e13-43e4-a1d4-ea27e403f385 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:12.676499 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0d754-9e13-43e4-a1d4-ea27e403f385 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:12.676776 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0d754-9e13-43e4-a1d4-ea27e403f385 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:12.677438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0d754-9e13-43e4-a1d4-ea27e403f385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:12.679433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0d754-9e13-43e4-a1d4-ea27e403f385 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:12.780388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1195/2392] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:45:12 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:45:12.784125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5079b81-3a28-43ef-9da9-52f4aed2530e admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:12.797221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5079b81-3a28-43ef-9da9-52f4aed2530e admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:12.801835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde1c71-47d3-480e-b3da-bd2ced92ddb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:12.802485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde1c71-47d3-480e-b3da-bd2ced92ddb6 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:12.802770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde1c71-47d3-480e-b3da-bd2ced92ddb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:12.803046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde1c71-47d3-480e-b3da-bd2ced92ddb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:12.803733 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde1c71-47d3-480e-b3da-bd2ced92ddb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:12.805323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde1c71-47d3-480e-b3da-bd2ced92ddb6 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:12.812994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:12.817964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1198/2393] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:45:12.822292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0d754-9e13-43e4-a1d4-ea27e403f385 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:12.833450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0d754-9e13-43e4-a1d4-ea27e403f385 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:12.838192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7426f9c-5529-4ef5-83f6-2996af9959f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:12.838946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7426f9c-5529-4ef5-83f6-2996af9959f3 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:12.839179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7426f9c-5529-4ef5-83f6-2996af9959f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:12.839456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7426f9c-5529-4ef5-83f6-2996af9959f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:12.840147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7426f9c-5529-4ef5-83f6-2996af9959f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:12.841786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7426f9c-5529-4ef5-83f6-2996af9959f3 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:12.851769 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1196/2394] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:45:12.857041 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde1c71-47d3-480e-b3da-bd2ced92ddb6 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:12.866727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde1c71-47d3-480e-b3da-bd2ced92ddb6 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:12.872583 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd852d8c-2805-4168-8fb1-8e73771bbfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:12.873237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd852d8c-2805-4168-8fb1-8e73771bbfd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:12.873523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd852d8c-2805-4168-8fb1-8e73771bbfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:12.873798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd852d8c-2805-4168-8fb1-8e73771bbfd3 admin admin] PATH_INFO: `/v3/projects/2ad8b5ba406a42c8bb9b2ddaa104794e/users/cd20d7480f29418eb1b04f78ff52c8ba/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:12.888298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd852d8c-2805-4168-8fb1-8e73771bbfd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8b5ba406a42c8bb9b2ddaa104794e, user_id=cd20d7480f29418eb1b04f78ff52c8ba, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:12.890990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd852d8c-2805-4168-8fb1-8e73771bbfd3 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:12.973702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1199/2395] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:45:12 2022] PUT /identity/v3/projects/2ad8b5ba406a42c8bb9b2ddaa104794e/users/cd20d7480f29418eb1b04f78ff52c8ba/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:45:12.986183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa56189-285c-408f-99e8-c8533df4bd7a None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:12.986546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa56189-285c-408f-99e8-c8533df4bd7a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:12.986835 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa56189-285c-408f-99e8-c8533df4bd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:13.015913 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0aa56189-285c-408f-99e8-c8533df4bd7a None None] MFA Rules not processed for user `cd20d7480f29418eb1b04f78ff52c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:13.017134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0aa56189-285c-408f-99e8-c8533df4bd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:13.063109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:13.084361 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1197/2396] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:13.089156 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd852d8c-2805-4168-8fb1-8e73771bbfd3 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:13.152143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd852d8c-2805-4168-8fb1-8e73771bbfd3 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:13.156966 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058b66c-fd51-46eb-8dbf-a7c4d63504d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:13.157581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b66c-fd51-46eb-8dbf-a7c4d63504d8 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:13.157860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b66c-fd51-46eb-8dbf-a7c4d63504d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:13.158166 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b66c-fd51-46eb-8dbf-a7c4d63504d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:13.158837 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058b66c-fd51-46eb-8dbf-a7c4d63504d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:13.160731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058b66c-fd51-46eb-8dbf-a7c4d63504d8 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:13.238161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1200/2397] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:45:13 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:45:13.242507 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa56189-285c-408f-99e8-c8533df4bd7a None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:13.253337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa56189-285c-408f-99e8-c8533df4bd7a None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:13.309505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdd260-4078-4bdc-9240-933ed9ed5873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:13.310212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd260-4078-4bdc-9240-933ed9ed5873 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:13.310491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd260-4078-4bdc-9240-933ed9ed5873 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:13.310761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd260-4078-4bdc-9240-933ed9ed5873 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:13.311504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdd260-4078-4bdc-9240-933ed9ed5873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:13.313392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdd260-4078-4bdc-9240-933ed9ed5873 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:13.380406 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1198/2398] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:45:13 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:45:13.384236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058b66c-fd51-46eb-8dbf-a7c4d63504d8 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:13.394904 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058b66c-fd51-46eb-8dbf-a7c4d63504d8 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:13.399690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039f11b-fe2f-4c52-98ce-6054bb296e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:13.400294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039f11b-fe2f-4c52-98ce-6054bb296e90 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:13.400579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039f11b-fe2f-4c52-98ce-6054bb296e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:13.401432 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039f11b-fe2f-4c52-98ce-6054bb296e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:13.402197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039f11b-fe2f-4c52-98ce-6054bb296e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:13.403783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039f11b-fe2f-4c52-98ce-6054bb296e90 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:13.415325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1201/2399] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:45:13.422073 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdd260-4078-4bdc-9240-933ed9ed5873 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:13.432327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdd260-4078-4bdc-9240-933ed9ed5873 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:13.437165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f9372-c193-4ead-a23d-174dc8f01547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:13.437833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f9372-c193-4ead-a23d-174dc8f01547 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:13.438155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f9372-c193-4ead-a23d-174dc8f01547 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:13.438434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f9372-c193-4ead-a23d-174dc8f01547 admin admin] PATH_INFO: `/v3/projects/fe5286137d394731805f61c508034c1c/users/445f0f7e7f554658ae7ca2af86f38987/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:13.454095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f9372-c193-4ead-a23d-174dc8f01547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5286137d394731805f61c508034c1c, user_id=445f0f7e7f554658ae7ca2af86f38987, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:13.457041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f9372-c193-4ead-a23d-174dc8f01547 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:13.536011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1199/2400] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:45:13 2022] PUT /identity/v3/projects/fe5286137d394731805f61c508034c1c/users/445f0f7e7f554658ae7ca2af86f38987/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:13.540112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039f11b-fe2f-4c52-98ce-6054bb296e90 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:13.550914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039f11b-fe2f-4c52-98ce-6054bb296e90 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:13.609905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c073e5-7a12-4708-b846-ad85445f524f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:13.610553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c073e5-7a12-4708-b846-ad85445f524f admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:13.610837 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c073e5-7a12-4708-b846-ad85445f524f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:13.611113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c073e5-7a12-4708-b846-ad85445f524f admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:13.611804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c073e5-7a12-4708-b846-ad85445f524f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:13.613438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c073e5-7a12-4708-b846-ad85445f524f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:13.627558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1202/2401] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:45:13.628751 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f9372-c193-4ead-a23d-174dc8f01547 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:13.639724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f9372-c193-4ead-a23d-174dc8f01547 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:13.644398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94300a79-010b-451e-957b-7419ab2e4d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:13.645053 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94300a79-010b-451e-957b-7419ab2e4d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:13.645332 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94300a79-010b-451e-957b-7419ab2e4d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:13.645606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94300a79-010b-451e-957b-7419ab2e4d1e admin admin] PATH_INFO: `/v3/projects/fe5286137d394731805f61c508034c1c/users/445f0f7e7f554658ae7ca2af86f38987/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:13.647601 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94300a79-010b-451e-957b-7419ab2e4d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5286137d394731805f61c508034c1c, user_id=445f0f7e7f554658ae7ca2af86f38987, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:13.650328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94300a79-010b-451e-957b-7419ab2e4d1e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:13.661074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1200/2402] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:45:13 2022] PUT /identity/v3/projects/fe5286137d394731805f61c508034c1c/users/445f0f7e7f554658ae7ca2af86f38987/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:13.672838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c9356-b673-4e3e-bb09-ddf9646af0eb None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:13.673148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c9356-b673-4e3e-bb09-ddf9646af0eb None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:13.673425 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c9356-b673-4e3e-bb09-ddf9646af0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:13.701462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-da3c9356-b673-4e3e-bb09-ddf9646af0eb None None] MFA Rules not processed for user `445f0f7e7f554658ae7ca2af86f38987`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:13.702707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-da3c9356-b673-4e3e-bb09-ddf9646af0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:13.735393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:13.772897 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1203/2403] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:45:13.784858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94300a79-010b-451e-957b-7419ab2e4d1e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:13.786669 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-94300a79-010b-451e-957b-7419ab2e4d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:13.846226 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94300a79-010b-451e-957b-7419ab2e4d1e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:13.850107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61689-b070-40ef-aec6-43b7ff5850a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:13.850731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61689-b070-40ef-aec6-43b7ff5850a6 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:13.851019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61689-b070-40ef-aec6-43b7ff5850a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:13.854914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61689-b070-40ef-aec6-43b7ff5850a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:13.902543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:13.911278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b61689-b070-40ef-aec6-43b7ff5850a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:13.912779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b61689-b070-40ef-aec6-43b7ff5850a6 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:13.921750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1201/2404] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:15.319724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0506a-ba6f-4d39-979d-428e25c73fc4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:15.320292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0506a-ba6f-4d39-979d-428e25c73fc4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:15.320292 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0506a-ba6f-4d39-979d-428e25c73fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:15.340585 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-2cc0506a-ba6f-4d39-979d-428e25c73fc4 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:15.341787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2cc0506a-ba6f-4d39-979d-428e25c73fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:15.400184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1204/2405] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:45:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:45:15.412772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61689-b070-40ef-aec6-43b7ff5850a6 service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:15.414280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-51b61689-b070-40ef-aec6-43b7ff5850a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:15.487089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61689-b070-40ef-aec6-43b7ff5850a6 service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:15.491988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8a347-e3ae-4575-ac4f-36481eb7e99f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:15.492603 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8a347-e3ae-4575-ac4f-36481eb7e99f service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:15.492884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8a347-e3ae-4575-ac4f-36481eb7e99f service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:15.493160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8a347-e3ae-4575-ac4f-36481eb7e99f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:15.502044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8a347-e3ae-4575-ac4f-36481eb7e99f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:15.503629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8a347-e3ae-4575-ac4f-36481eb7e99f service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:15.512003 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1202/2406] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:45:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:45:16.537636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0506a-ba6f-4d39-979d-428e25c73fc4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:16.539134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2cc0506a-ba6f-4d39-979d-428e25c73fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:16.600627 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0506a-ba6f-4d39-979d-428e25c73fc4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:16.604967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fc089-ff52-46cb-b8ab-83bf62aa1cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:16.605624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc089-ff52-46cb-b8ab-83bf62aa1cbc service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:16.605908 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc089-ff52-46cb-b8ab-83bf62aa1cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:16.606209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc089-ff52-46cb-b8ab-83bf62aa1cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:16.608116 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4d1fc089-ff52-46cb-b8ab-83bf62aa1cbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:16.690272 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fc089-ff52-46cb-b8ab-83bf62aa1cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:16.691834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fc089-ff52-46cb-b8ab-83bf62aa1cbc service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:16.704457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1205/2407] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:16.932992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8a347-e3ae-4575-ac4f-36481eb7e99f service swift] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:16.944059 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8a347-e3ae-4575-ac4f-36481eb7e99f service swift] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:16.948366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd28a9-facf-4c2f-8c33-6951529118ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:16.948980 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd28a9-facf-4c2f-8c33-6951529118ad service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:16.949260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd28a9-facf-4c2f-8c33-6951529118ad service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:16.949702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd28a9-facf-4c2f-8c33-6951529118ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:17.009870 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd28a9-facf-4c2f-8c33-6951529118ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:17.011978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd28a9-facf-4c2f-8c33-6951529118ad service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:17.022060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1203/2408] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:17.912097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fc089-ff52-46cb-b8ab-83bf62aa1cbc service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:17.913594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4d1fc089-ff52-46cb-b8ab-83bf62aa1cbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:17.926368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fc089-ff52-46cb-b8ab-83bf62aa1cbc service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:17.930843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3391947-f7d5-4def-9ddc-9e87b2e232be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:17.931472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3391947-f7d5-4def-9ddc-9e87b2e232be service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:17.936399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3391947-f7d5-4def-9ddc-9e87b2e232be service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:17.936399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3391947-f7d5-4def-9ddc-9e87b2e232be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:17.936399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b3391947-f7d5-4def-9ddc-9e87b2e232be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:17.948878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3391947-f7d5-4def-9ddc-9e87b2e232be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:17.950569 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3391947-f7d5-4def-9ddc-9e87b2e232be service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:17.963841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1206/2409] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:18.385350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd28a9-facf-4c2f-8c33-6951529118ad service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:18.394593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd28a9-facf-4c2f-8c33-6951529118ad service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:18.398645 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0819b7-767d-4858-a809-5b9ed23ace01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:18.399265 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0819b7-767d-4858-a809-5b9ed23ace01 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:18.399547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0819b7-767d-4858-a809-5b9ed23ace01 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:18.399829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0819b7-767d-4858-a809-5b9ed23ace01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:18.409965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0819b7-767d-4858-a809-5b9ed23ace01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:18.411500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0819b7-767d-4858-a809-5b9ed23ace01 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:18.423389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1204/2410] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:24.208639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3391947-f7d5-4def-9ddc-9e87b2e232be service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:24.214260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:24.329335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3391947-f7d5-4def-9ddc-9e87b2e232be service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:24.329335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:24.329335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:24.329335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:24.329761 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] PATH_INFO: `/v3/users/276f842586c34e509e5e87058c725177` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:24.329761 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] RBAC: Authorizing `identity:delete_user(user_id=276f842586c34e509e5e87058c725177)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:24.329761 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:24.625033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '276f842586c34e509e5e87058c725177'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:24.722947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '276f842586c34e509e5e87058c725177'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:24.737370 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '276f842586c34e509e5e87058c725177'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:24.738124 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '276f842586c34e509e5e87058c725177'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:24.747919 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '276f842586c34e509e5e87058c725177'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:24.755434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1207/2411] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:45:24 2022] DELETE /identity/v3/users/276f842586c34e509e5e87058c725177 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:24.760590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d0f2e-de46-4798-aaa4-f1fa2f7e6d9c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:24.762575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d0f2e-de46-4798-aaa4-f1fa2f7e6d9c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:24.762575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d0f2e-de46-4798-aaa4-f1fa2f7e6d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:24.781778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0b3d0f2e-de46-4798-aaa4-f1fa2f7e6d9c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:24.783261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0b3d0f2e-de46-4798-aaa4-f1fa2f7e6d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:24.849063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1205/2412] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:24.860126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:24.861570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:24.925717 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d534bb-350d-453f-afa1-c2cc832dcfef admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:24.929503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746995b3-8897-4d42-83a7-263fb62e0d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:24.930266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746995b3-8897-4d42-83a7-263fb62e0d05 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:24.930545 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746995b3-8897-4d42-83a7-263fb62e0d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:24.930815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746995b3-8897-4d42-83a7-263fb62e0d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:24.941358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746995b3-8897-4d42-83a7-263fb62e0d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:24.942800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746995b3-8897-4d42-83a7-263fb62e0d05 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:24.953462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1208/2413] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:25.583884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d0f2e-de46-4798-aaa4-f1fa2f7e6d9c None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:25.586666 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0b3d0f2e-de46-4798-aaa4-f1fa2f7e6d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:25.601284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d0f2e-de46-4798-aaa4-f1fa2f7e6d9c None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:25.607793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:25.608448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:25.608728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:25.609016 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] PATH_INFO: `/v3/projects/3820366e057b4c50a74ce1b871575c90` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:25.610393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3820366e057b4c50a74ce1b871575c90)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:25.612306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:25.613962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:25.752733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3820366e057b4c50a74ce1b871575c90', 'request_id': 'req-a4ce15c4-e640-44c5-8743-751ccecea9dd'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:25.759183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3820366e057b4c50a74ce1b871575c90', 'request_id': 'req-a4ce15c4-e640-44c5-8743-751ccecea9dd'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:25.824939 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3820366e057b4c50a74ce1b871575c90', 'request_id': 'req-a4ce15c4-e640-44c5-8743-751ccecea9dd'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:25.836034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] The token cache is being invalidate because project 3820366e057b4c50a74ce1b871575c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:45:25.836325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:25.836969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:25.838889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1206/2414] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:45:25 2022] DELETE /identity/v3/projects/3820366e057b4c50a74ce1b871575c90 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:25.843492 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746995b3-8897-4d42-83a7-263fb62e0d05 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:25.853396 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746995b3-8897-4d42-83a7-263fb62e0d05 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:25.879602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:25.914995 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:25.915960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:25.916983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:25.917259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] PATH_INFO: `/v3/users/cd20d7480f29418eb1b04f78ff52c8ba` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:25.919029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd20d7480f29418eb1b04f78ff52c8ba)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:25.920953 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:26.095655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd20d7480f29418eb1b04f78ff52c8ba'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:26.185400 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd20d7480f29418eb1b04f78ff52c8ba'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:26.198981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd20d7480f29418eb1b04f78ff52c8ba'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:26.199799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd20d7480f29418eb1b04f78ff52c8ba'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:26.207296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd20d7480f29418eb1b04f78ff52c8ba'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:26.214183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1209/2415] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:45:25 2022] DELETE /identity/v3/users/cd20d7480f29418eb1b04f78ff52c8ba => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:26.632592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:26.634294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:26.648654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce15c4-e640-44c5-8743-751ccecea9dd admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:26.746068 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:26.746869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:26.747231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:26.747584 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] PATH_INFO: `/v3/projects/2ad8b5ba406a42c8bb9b2ddaa104794e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:26.749374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ad8b5ba406a42c8bb9b2ddaa104794e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:26.751444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:26.900229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ad8b5ba406a42c8bb9b2ddaa104794e', 'request_id': 'req-8560d1e0-f311-4898-9717-626af2093414'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:26.907287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ad8b5ba406a42c8bb9b2ddaa104794e', 'request_id': 'req-8560d1e0-f311-4898-9717-626af2093414'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:26.979672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ad8b5ba406a42c8bb9b2ddaa104794e', 'request_id': 'req-8560d1e0-f311-4898-9717-626af2093414'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:26.990843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] The token cache is being invalidate because project 2ad8b5ba406a42c8bb9b2ddaa104794e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:45:26.991195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:26.991847 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:26.993614 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1207/2416] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:45:26 2022] DELETE /identity/v3/projects/2ad8b5ba406a42c8bb9b2ddaa104794e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:26.997979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:26.999519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:27.012426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5179a1-0edb-4afb-bb1d-86dadea6e07a admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:27.072324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11916e-b64b-4580-af59-d2e13a8eaf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:27.073476 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11916e-b64b-4580-af59-d2e13a8eaf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:27.073796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11916e-b64b-4580-af59-d2e13a8eaf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:27.074035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11916e-b64b-4580-af59-d2e13a8eaf3e admin admin] PATH_INFO: `/v3/users/445f0f7e7f554658ae7ca2af86f38987` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:27.075403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11916e-b64b-4580-af59-d2e13a8eaf3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=445f0f7e7f554658ae7ca2af86f38987)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:27.077309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11916e-b64b-4580-af59-d2e13a8eaf3e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:27.233434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9e11916e-b64b-4580-af59-d2e13a8eaf3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '445f0f7e7f554658ae7ca2af86f38987'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:27.235085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:27.328320 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9e11916e-b64b-4580-af59-d2e13a8eaf3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '445f0f7e7f554658ae7ca2af86f38987'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:27.342152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9e11916e-b64b-4580-af59-d2e13a8eaf3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '445f0f7e7f554658ae7ca2af86f38987'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:27.342809 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9e11916e-b64b-4580-af59-d2e13a8eaf3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '445f0f7e7f554658ae7ca2af86f38987'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:27.349962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-9e11916e-b64b-4580-af59-d2e13a8eaf3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '445f0f7e7f554658ae7ca2af86f38987'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:27.358491 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1210/2417] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:45:26 2022] DELETE /identity/v3/users/445f0f7e7f554658ae7ca2af86f38987 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:27.775546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:27.779671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:27.790788 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560d1e0-f311-4898-9717-626af2093414 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:27.810140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:27.851453 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:27.852132 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:27.852420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:27.852698 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] PATH_INFO: `/v3/projects/fe5286137d394731805f61c508034c1c` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:27.854560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5286137d394731805f61c508034c1c)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:27.856536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:27.954617 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5286137d394731805f61c508034c1c', 'request_id': 'req-f0c753f3-aeac-4139-8ee8-89567c84d05a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:27.969989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5286137d394731805f61c508034c1c', 'request_id': 'req-f0c753f3-aeac-4139-8ee8-89567c84d05a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:28.043045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5286137d394731805f61c508034c1c', 'request_id': 'req-f0c753f3-aeac-4139-8ee8-89567c84d05a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:28.053476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] The token cache is being invalidate because project fe5286137d394731805f61c508034c1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:45:28.053769 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:28.054440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0c753f3-aeac-4139-8ee8-89567c84d05a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:28.055962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1208/2418] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:45:27 2022] DELETE /identity/v3/projects/fe5286137d394731805f61c508034c1c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:28.061613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9ef88-7225-4895-b5cd-93974c718d5b None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:28.072698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9ef88-7225-4895-b5cd-93974c718d5b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:28.072698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9ef88-7225-4895-b5cd-93974c718d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:28.082258 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-8cb9ef88-7225-4895-b5cd-93974c718d5b None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:28.083467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8cb9ef88-7225-4895-b5cd-93974c718d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:28.141935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1211/2419] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:28.150119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f814d-03ef-4bfb-9de6-46f1c08e7e58 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:28.150413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f814d-03ef-4bfb-9de6-46f1c08e7e58 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:28.150751 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f814d-03ef-4bfb-9de6-46f1c08e7e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:28.172079 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-f64f814d-03ef-4bfb-9de6-46f1c08e7e58 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:28.173334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f64f814d-03ef-4bfb-9de6-46f1c08e7e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:28.181288 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1209/2420] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:45:28.186286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9ef88-7225-4895-b5cd-93974c718d5b None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:28.195865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9ef88-7225-4895-b5cd-93974c718d5b None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:28.200673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199bc23-13c0-4c06-8593-1f3527dde077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:28.201300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199bc23-13c0-4c06-8593-1f3527dde077 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:28.201574 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199bc23-13c0-4c06-8593-1f3527dde077 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:28.201846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199bc23-13c0-4c06-8593-1f3527dde077 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:28.202572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199bc23-13c0-4c06-8593-1f3527dde077 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:45:28.202878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199bc23-13c0-4c06-8593-1f3527dde077 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:28.204451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199bc23-13c0-4c06-8593-1f3527dde077 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:28.216553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1212/2421] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:45:28.220958 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f814d-03ef-4bfb-9de6-46f1c08e7e58 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:28.232096 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f814d-03ef-4bfb-9de6-46f1c08e7e58 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:28.235589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9d6fd-e060-4287-b6d7-a709864c3a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:28.236190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9d6fd-e060-4287-b6d7-a709864c3a72 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:28.236469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9d6fd-e060-4287-b6d7-a709864c3a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:28.236742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9d6fd-e060-4287-b6d7-a709864c3a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:28.237402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9d6fd-e060-4287-b6d7-a709864c3a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:28.239368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9d6fd-e060-4287-b6d7-a709864c3a72 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:28.312919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1210/2422] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:45:28 2022] POST /identity/v3/projects => generated 397 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:45:28.317029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199bc23-13c0-4c06-8593-1f3527dde077 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:28.327681 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199bc23-13c0-4c06-8593-1f3527dde077 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:28.382364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1edf9c-a0ee-49fa-9baf-e5f7e7a3ddee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:28.382971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1edf9c-a0ee-49fa-9baf-e5f7e7a3ddee admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:28.383252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1edf9c-a0ee-49fa-9baf-e5f7e7a3ddee admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:28.383524 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1edf9c-a0ee-49fa-9baf-e5f7e7a3ddee admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:28.384184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1edf9c-a0ee-49fa-9baf-e5f7e7a3ddee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:28.386942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1edf9c-a0ee-49fa-9baf-e5f7e7a3ddee admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:28.448634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1213/2423] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:45:28 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:45:28.453052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9d6fd-e060-4287-b6d7-a709864c3a72 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:28.463380 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9d6fd-e060-4287-b6d7-a709864c3a72 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:28.467969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d850c1-8e4b-4a8c-8c68-23de8203f560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:28.472357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d850c1-8e4b-4a8c-8c68-23de8203f560 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:28.472357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d850c1-8e4b-4a8c-8c68-23de8203f560 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:28.472357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d850c1-8e4b-4a8c-8c68-23de8203f560 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:28.472357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d850c1-8e4b-4a8c-8c68-23de8203f560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:28.472357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d850c1-8e4b-4a8c-8c68-23de8203f560 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:28.481845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1211/2424] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:45:28.485889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1edf9c-a0ee-49fa-9baf-e5f7e7a3ddee admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:28.496643 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1edf9c-a0ee-49fa-9baf-e5f7e7a3ddee admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:28.501219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164633d4-9d11-4239-b8a3-8c4578bc432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:28.501814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164633d4-9d11-4239-b8a3-8c4578bc432a admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:28.502122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164633d4-9d11-4239-b8a3-8c4578bc432a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:28.502401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164633d4-9d11-4239-b8a3-8c4578bc432a admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:28.503085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164633d4-9d11-4239-b8a3-8c4578bc432a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:28.504659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164633d4-9d11-4239-b8a3-8c4578bc432a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:28.515881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1214/2425] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:45:28.520147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d850c1-8e4b-4a8c-8c68-23de8203f560 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:28.530445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d850c1-8e4b-4a8c-8c68-23de8203f560 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:28.535084 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894048cc-8c26-4f60-bab0-845f0642925d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:28.535742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894048cc-8c26-4f60-bab0-845f0642925d admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:28.536025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894048cc-8c26-4f60-bab0-845f0642925d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:28.536302 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894048cc-8c26-4f60-bab0-845f0642925d admin admin] PATH_INFO: `/v3/projects/aa2506bb406d4d9386488b575d91a420/users/b23d0b3d6ea8446cb7e30c323065be8a/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:28.550702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894048cc-8c26-4f60-bab0-845f0642925d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa2506bb406d4d9386488b575d91a420, user_id=b23d0b3d6ea8446cb7e30c323065be8a, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:28.553347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894048cc-8c26-4f60-bab0-845f0642925d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:28.642246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1212/2426] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:45:28 2022] PUT /identity/v3/projects/aa2506bb406d4d9386488b575d91a420/users/b23d0b3d6ea8446cb7e30c323065be8a/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:28.653849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376f364-11dd-4c16-86a6-070d80b356af None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:28.654184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376f364-11dd-4c16-86a6-070d80b356af None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:28.654463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376f364-11dd-4c16-86a6-070d80b356af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:28.682728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f376f364-11dd-4c16-86a6-070d80b356af None None] MFA Rules not processed for user `b23d0b3d6ea8446cb7e30c323065be8a`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:28.684132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f376f364-11dd-4c16-86a6-070d80b356af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:28.702162 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:28.748950 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1215/2427] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:28 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:28.760196 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894048cc-8c26-4f60-bab0-845f0642925d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:28.761875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-894048cc-8c26-4f60-bab0-845f0642925d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:28.814826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:28.819966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894048cc-8c26-4f60-bab0-845f0642925d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:28.823849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233962f-a9fb-4435-8833-ecdbb3cdccbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:28.826852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233962f-a9fb-4435-8833-ecdbb3cdccbe service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:28.826852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233962f-a9fb-4435-8833-ecdbb3cdccbe service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:28.826852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233962f-a9fb-4435-8833-ecdbb3cdccbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:28.834222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233962f-a9fb-4435-8833-ecdbb3cdccbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:28.835741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233962f-a9fb-4435-8833-ecdbb3cdccbe service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:28.845270 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1213/2428] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:28 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:32.573060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376f364-11dd-4c16-86a6-070d80b356af None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:32.574802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f376f364-11dd-4c16-86a6-070d80b356af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:32.633165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376f364-11dd-4c16-86a6-070d80b356af None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:32.637262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438939e9-597b-4511-a358-4610dd1a927e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:32.637873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438939e9-597b-4511-a358-4610dd1a927e service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:32.638187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438939e9-597b-4511-a358-4610dd1a927e service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:32.638460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438939e9-597b-4511-a358-4610dd1a927e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:32.640399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-438939e9-597b-4511-a358-4610dd1a927e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:32.701979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438939e9-597b-4511-a358-4610dd1a927e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:32.703439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438939e9-597b-4511-a358-4610dd1a927e service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:32.712417 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1216/2429] 10.222.0.73 () {60 vars in 1396 bytes} [Thu Aug 4 21:45:32 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:38.894175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:39.953121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233962f-a9fb-4435-8833-ecdbb3cdccbe service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:40.018333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233962f-a9fb-4435-8833-ecdbb3cdccbe service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:40.024019 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:40.024662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:40.025055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:40.025331 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] PATH_INFO: `/v3/users/b23d0b3d6ea8446cb7e30c323065be8a` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:40.027119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23d0b3d6ea8446cb7e30c323065be8a)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:40.028706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:40.219593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23d0b3d6ea8446cb7e30c323065be8a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:40.299265 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23d0b3d6ea8446cb7e30c323065be8a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:40.299890 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23d0b3d6ea8446cb7e30c323065be8a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:40.313349 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23d0b3d6ea8446cb7e30c323065be8a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:40.321300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23d0b3d6ea8446cb7e30c323065be8a'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:40.327912 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1214/2430] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:45:39 2022] DELETE /identity/v3/users/b23d0b3d6ea8446cb7e30c323065be8a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:40.333302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28e06f-c7a2-4b5f-b7c0-47a2dbac98f6 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:40.333302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28e06f-c7a2-4b5f-b7c0-47a2dbac98f6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:40.333864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28e06f-c7a2-4b5f-b7c0-47a2dbac98f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:40.353388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-db28e06f-c7a2-4b5f-b7c0-47a2dbac98f6 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:40.355380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-db28e06f-c7a2-4b5f-b7c0-47a2dbac98f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:40.418277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1217/2431] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:40.428586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:40.430219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:40.488868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e85c7-2cd8-4146-a406-f38470fe8a56 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:40.496343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5c4ee-c50f-4cb4-b948-2a1fb774ef05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:40.496975 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5c4ee-c50f-4cb4-b948-2a1fb774ef05 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:40.497288 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5c4ee-c50f-4cb4-b948-2a1fb774ef05 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:40.497564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5c4ee-c50f-4cb4-b948-2a1fb774ef05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:40.507046 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5c4ee-c50f-4cb4-b948-2a1fb774ef05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:40.508485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5c4ee-c50f-4cb4-b948-2a1fb774ef05 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:40.518652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1215/2432] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:45:40.895749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28e06f-c7a2-4b5f-b7c0-47a2dbac98f6 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:40.897185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-db28e06f-c7a2-4b5f-b7c0-47a2dbac98f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:40.909995 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28e06f-c7a2-4b5f-b7c0-47a2dbac98f6 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:40.915007 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:40.916926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:40.917988 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:40.918300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] PATH_INFO: `/v3/projects/aa2506bb406d4d9386488b575d91a420` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:40.919552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa2506bb406d4d9386488b575d91a420)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:40.922061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:40.935946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:41.016880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa2506bb406d4d9386488b575d91a420', 'request_id': 'req-43f0e584-7bf8-42e6-bcee-3eb985e44749'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:41.088845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa2506bb406d4d9386488b575d91a420', 'request_id': 'req-43f0e584-7bf8-42e6-bcee-3eb985e44749'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:41.095210 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa2506bb406d4d9386488b575d91a420', 'request_id': 'req-43f0e584-7bf8-42e6-bcee-3eb985e44749'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:41.107936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] The token cache is being invalidate because project aa2506bb406d4d9386488b575d91a420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:45:41.108227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:41.108917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:45:41.110659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1218/2433] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:45:40 2022] DELETE /identity/v3/projects/aa2506bb406d4d9386488b575d91a420 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:45:44.635964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62b16a-a4a7-4628-88ea-56549a52f357 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:44.636428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62b16a-a4a7-4628-88ea-56549a52f357 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:44.636557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62b16a-a4a7-4628-88ea-56549a52f357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:44.655569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-bd62b16a-a4a7-4628-88ea-56549a52f357 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:45:44.656810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bd62b16a-a4a7-4628-88ea-56549a52f357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:44.673849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:44.713987 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1216/2434] 10.222.0.73 () {58 vars in 967 bytes} [Thu Aug 4 21:45:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:45:44.726809 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:44.728230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:44.797863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0e584-7bf8-42e6-bcee-3eb985e44749 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:44.803306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05087a61-9326-4709-92d2-60561f31ece6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:44.804389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05087a61-9326-4709-92d2-60561f31ece6 service swift] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:44.804389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05087a61-9326-4709-92d2-60561f31ece6 service swift] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:44.804565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05087a61-9326-4709-92d2-60561f31ece6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:44.814287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05087a61-9326-4709-92d2-60561f31ece6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:44.815876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05087a61-9326-4709-92d2-60561f31ece6 service swift] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:44.824424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1219/2435] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:45:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:45:54.807835 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=15 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:54.821236 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:45:57.560147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62b16a-a4a7-4628-88ea-56549a52f357 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:45:57.561578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bd62b16a-a4a7-4628-88ea-56549a52f357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:57.635604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62b16a-a4a7-4628-88ea-56549a52f357 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:45:57.641262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7cac1-3e3e-43d4-b8ce-2f33e6073412 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:45:57.641871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7cac1-3e3e-43d4-b8ce-2f33e6073412 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:45:57.642182 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7cac1-3e3e-43d4-b8ce-2f33e6073412 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:45:57.642452 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7cac1-3e3e-43d4-b8ce-2f33e6073412 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:45:57.644156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4af7cac1-3e3e-43d4-b8ce-2f33e6073412 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:45:57.703355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7cac1-3e3e-43d4-b8ce-2f33e6073412 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:45:57.705002 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7cac1-3e3e-43d4-b8ce-2f33e6073412 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:45:57.714040 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1217/2436] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:45:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:46:07.585628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05087a61-9326-4709-92d2-60561f31ece6 service swift] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:46:07.587317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-05087a61-9326-4709-92d2-60561f31ece6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:46:07.645269 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05087a61-9326-4709-92d2-60561f31ece6 service swift] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:46:07.649096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe97b2-0f4c-4f3a-96e0-73755eaffd05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:46:07.649778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe97b2-0f4c-4f3a-96e0-73755eaffd05 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:46:07.650047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe97b2-0f4c-4f3a-96e0-73755eaffd05 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:46:07.650322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe97b2-0f4c-4f3a-96e0-73755eaffd05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:46:07.652082 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7afe97b2-0f4c-4f3a-96e0-73755eaffd05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:46:07.661697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe97b2-0f4c-4f3a-96e0-73755eaffd05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:46:07.663149 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe97b2-0f4c-4f3a-96e0-73755eaffd05 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:46:07.670949 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1220/2437] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:46:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:46:07.709846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=56 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:46:08.739500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7cac1-3e3e-43d4-b8ce-2f33e6073412 service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:46:08.751180 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7cac1-3e3e-43d4-b8ce-2f33e6073412 service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:46:08.760081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab246f-142e-47b2-9e95-b9193b189e9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:46:08.760723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab246f-142e-47b2-9e95-b9193b189e9c service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:46:08.761009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab246f-142e-47b2-9e95-b9193b189e9c service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:46:08.761286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab246f-142e-47b2-9e95-b9193b189e9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:46:08.768429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bfab246f-142e-47b2-9e95-b9193b189e9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:46:08.833706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab246f-142e-47b2-9e95-b9193b189e9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:46:08.835374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab246f-142e-47b2-9e95-b9193b189e9c service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:46:08.845494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1218/2438] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:46:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:46:17.667387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=26 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:46:18.840737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=28 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:46:27.558939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04416b8-8382-4032-8a07-175fe1caf48e None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:46:27.559520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04416b8-8382-4032-8a07-175fe1caf48e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:46:27.559520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04416b8-8382-4032-8a07-175fe1caf48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:46:27.578430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-b04416b8-8382-4032-8a07-175fe1caf48e None None] MFA Rules not processed for user `d9bc42bec393430b9b3e9399a8c4f95e`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:46:27.579626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b04416b8-8382-4032-8a07-175fe1caf48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:46:27.586933 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1221/2439] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:46:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:46:34.321188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab246f-142e-47b2-9e95-b9193b189e9c service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:46:34.323893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-bfab246f-142e-47b2-9e95-b9193b189e9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:46:34.340647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab246f-142e-47b2-9e95-b9193b189e9c service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:46:34.346915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be78ca3-40b9-4a8c-9e62-105e8952bd4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:46:34.347745 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be78ca3-40b9-4a8c-9e62-105e8952bd4f service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:46:34.348059 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be78ca3-40b9-4a8c-9e62-105e8952bd4f service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:46:34.348416 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be78ca3-40b9-4a8c-9e62-105e8952bd4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:46:34.350385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5be78ca3-40b9-4a8c-9e62-105e8952bd4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:46:34.364149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be78ca3-40b9-4a8c-9e62-105e8952bd4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:46:34.365889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be78ca3-40b9-4a8c-9e62-105e8952bd4f service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:46:34.378391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1219/2440] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:46:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:46:37.573681 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:46:44.373822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=6 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:47:00.988267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04416b8-8382-4032-8a07-175fe1caf48e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:47:01.004741 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04416b8-8382-4032-8a07-175fe1caf48e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:47:01.011033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a27072-3456-460d-857e-18ae0a9b69b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:47:01.011668 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a27072-3456-460d-857e-18ae0a9b69b4 service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:47:01.011938 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a27072-3456-460d-857e-18ae0a9b69b4 service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:47:01.012217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a27072-3456-460d-857e-18ae0a9b69b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:47:01.014214 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-65a27072-3456-460d-857e-18ae0a9b69b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:47:01.031809 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a27072-3456-460d-857e-18ae0a9b69b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:47:01.033413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a27072-3456-460d-857e-18ae0a9b69b4 service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:47:01.048175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1222/2441] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:47:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:47:06.563118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7b0b5-e093-4096-b9f8-42b70c77965e None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:47:06.567610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7b0b5-e093-4096-b9f8-42b70c77965e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:47:06.567610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7b0b5-e093-4096-b9f8-42b70c77965e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:47:06.584091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1bf7b0b5-e093-4096-b9f8-42b70c77965e None None] MFA Rules not processed for user `95fb2edf82644e7f8ce461fcbb52007b`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:47:06.585839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1bf7b0b5-e093-4096-b9f8-42b70c77965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:47:06.641779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1220/2442] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:47:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:47:06.958618 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a27072-3456-460d-857e-18ae0a9b69b4 service nova] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:47:06.968605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a27072-3456-460d-857e-18ae0a9b69b4 service nova] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:47:06.972338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee345f3b-d7a3-4744-9008-7b33cc10bfc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:47:06.973031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee345f3b-d7a3-4744-9008-7b33cc10bfc1 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:47:06.973313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee345f3b-d7a3-4744-9008-7b33cc10bfc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:47:06.973587 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee345f3b-d7a3-4744-9008-7b33cc10bfc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:47:06.975651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ee345f3b-d7a3-4744-9008-7b33cc10bfc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:47:06.986096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee345f3b-d7a3-4744-9008-7b33cc10bfc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:47:06.987611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee345f3b-d7a3-4744-9008-7b33cc10bfc1 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:47:06.997587 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1223/2443] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:47:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:47:07.392333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7b0b5-e093-4096-b9f8-42b70c77965e None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:47:07.393828 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1bf7b0b5-e093-4096-b9f8-42b70c77965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:47:07.464815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7b0b5-e093-4096-b9f8-42b70c77965e None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:47:07.469876 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8feb6-94c3-40f9-b8a0-05a5dbe2c360 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:47:07.470453 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8feb6-94c3-40f9-b8a0-05a5dbe2c360 service placement] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:47:07.470737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8feb6-94c3-40f9-b8a0-05a5dbe2c360 service placement] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:47:07.471013 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8feb6-94c3-40f9-b8a0-05a5dbe2c360 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:47:07.472796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5fd8feb6-94c3-40f9-b8a0-05a5dbe2c360 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:47:07.484556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8feb6-94c3-40f9-b8a0-05a5dbe2c360 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:47:07.486186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8feb6-94c3-40f9-b8a0-05a5dbe2c360 service placement] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:47:07.497306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1221/2444] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:47:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:47:16.992702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=12 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:47:17.492728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=57 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:47:19.756101 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee345f3b-d7a3-4744-9008-7b33cc10bfc1 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:47:19.769240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee345f3b-d7a3-4744-9008-7b33cc10bfc1 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:47:19.775728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c7716-4bd4-4822-80bb-3e0c24eaf33d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:47:19.776379 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c7716-4bd4-4822-80bb-3e0c24eaf33d service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:47:19.779390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c7716-4bd4-4822-80bb-3e0c24eaf33d service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:47:19.779390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c7716-4bd4-4822-80bb-3e0c24eaf33d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:47:19.779390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-963c7716-4bd4-4822-80bb-3e0c24eaf33d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:47:19.791424 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c7716-4bd4-4822-80bb-3e0c24eaf33d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:47:19.796567 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c7716-4bd4-4822-80bb-3e0c24eaf33d service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:47:19.807382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1224/2445] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:47:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:47:29.799101 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=6 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:47:47.465319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916023d-72f4-41f5-a560-d21f5777608e None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:47:47.465854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916023d-72f4-41f5-a560-d21f5777608e None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:47:47.465983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916023d-72f4-41f5-a560-d21f5777608e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:47:47.485840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1916023d-72f4-41f5-a560-d21f5777608e None None] MFA Rules not processed for user `a23fcabad2354e7ab51a22b412bf11e6`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:47:47.487128 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1916023d-72f4-41f5-a560-d21f5777608e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:47:47.495058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1222/2446] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:47:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:47:47.826295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c7716-4bd4-4822-80bb-3e0c24eaf33d service nova] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:47:47.827774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-963c7716-4bd4-4822-80bb-3e0c24eaf33d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:47:47.839500 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c7716-4bd4-4822-80bb-3e0c24eaf33d service nova] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:47:47.843184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d81dc-de52-41f3-84ee-835763d65507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:47:47.843787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d81dc-de52-41f3-84ee-835763d65507 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:47:47.844063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d81dc-de52-41f3-84ee-835763d65507 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:47:47.844354 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d81dc-de52-41f3-84ee-835763d65507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:47:47.846108 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-da0d81dc-de52-41f3-84ee-835763d65507 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:47:47.856109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d81dc-de52-41f3-84ee-835763d65507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:47:47.857501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d81dc-de52-41f3-84ee-835763d65507 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:47:47.865301 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1225/2447] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:47:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:47:57.480374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:47:57.861798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=6 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:04.984390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916023d-72f4-41f5-a560-d21f5777608e None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:05.001624 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916023d-72f4-41f5-a560-d21f5777608e None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:05.005565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a27f71-eb7b-4e49-887f-da87f7d0730d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:05.006233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a27f71-eb7b-4e49-887f-da87f7d0730d service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:05.006520 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a27f71-eb7b-4e49-887f-da87f7d0730d service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:05.006796 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a27f71-eb7b-4e49-887f-da87f7d0730d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:05.008590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-04a27f71-eb7b-4e49-887f-da87f7d0730d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:05.080884 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a27f71-eb7b-4e49-887f-da87f7d0730d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:05.082290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a27f71-eb7b-4e49-887f-da87f7d0730d service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:05.092610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1223/2448] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:09.123866 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d81dc-de52-41f3-84ee-835763d65507 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:09.136554 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d81dc-de52-41f3-84ee-835763d65507 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:09.141958 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4a637-d38a-4027-bee2-9fc777c9bd56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:09.142677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a637-d38a-4027-bee2-9fc777c9bd56 service placement] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:09.143033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a637-d38a-4027-bee2-9fc777c9bd56 service placement] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:09.143377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a637-d38a-4027-bee2-9fc777c9bd56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:09.145238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1dd4a637-d38a-4027-bee2-9fc777c9bd56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:09.164576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4a637-d38a-4027-bee2-9fc777c9bd56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:09.166205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4a637-d38a-4027-bee2-9fc777c9bd56 service placement] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:09.178770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1226/2449] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:09.261220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a27f71-eb7b-4e49-887f-da87f7d0730d service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:09.261220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-04a27f71-eb7b-4e49-887f-da87f7d0730d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:09.272619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a27f71-eb7b-4e49-887f-da87f7d0730d service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:09.277444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b605e7-4995-4f50-9631-260c00f24647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:09.278162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b605e7-4995-4f50-9631-260c00f24647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:09.278483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b605e7-4995-4f50-9631-260c00f24647 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:09.278767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b605e7-4995-4f50-9631-260c00f24647 admin admin] PATH_INFO: `/v3/users/d9bc42bec393430b9b3e9399a8c4f95e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:09.279953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b605e7-4995-4f50-9631-260c00f24647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9bc42bec393430b9b3e9399a8c4f95e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:09.281862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b605e7-4995-4f50-9631-260c00f24647 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:09.437431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-63b605e7-4995-4f50-9631-260c00f24647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9bc42bec393430b9b3e9399a8c4f95e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:09.526072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-63b605e7-4995-4f50-9631-260c00f24647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9bc42bec393430b9b3e9399a8c4f95e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:09.526923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-63b605e7-4995-4f50-9631-260c00f24647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9bc42bec393430b9b3e9399a8c4f95e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:09.542931 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-63b605e7-4995-4f50-9631-260c00f24647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9bc42bec393430b9b3e9399a8c4f95e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:09.549876 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-63b605e7-4995-4f50-9631-260c00f24647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9bc42bec393430b9b3e9399a8c4f95e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:09.555959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1224/2450] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:48:09 2022] DELETE /identity/v3/users/d9bc42bec393430b9b3e9399a8c4f95e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:48:09.818508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4a637-d38a-4027-bee2-9fc777c9bd56 service placement] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:09.829453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4a637-d38a-4027-bee2-9fc777c9bd56 service placement] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:09.888239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:09.888882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:09.889167 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:09.889441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] PATH_INFO: `/v3/projects/26fb895147494be98bb6ceb8704984b7` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:09.890625 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26fb895147494be98bb6ceb8704984b7)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:09.892518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:09.994170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26fb895147494be98bb6ceb8704984b7', 'request_id': 'req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:10.051028 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26fb895147494be98bb6ceb8704984b7', 'request_id': 'req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:10.057251 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26fb895147494be98bb6ceb8704984b7', 'request_id': 'req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:10.069170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] The token cache is being invalidate because project 26fb895147494be98bb6ceb8704984b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:48:10.069468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:10.070153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-37a5b981-d0a2-4524-a7c7-f1da9962b0f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:10.071669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1227/2451] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:48:09 2022] DELETE /identity/v3/projects/26fb895147494be98bb6ceb8704984b7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:10.076741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f40f4-1030-403b-b9f6-de633b04bf00 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:10.077037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f40f4-1030-403b-b9f6-de633b04bf00 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:10.077317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f40f4-1030-403b-b9f6-de633b04bf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:10.099150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c11f40f4-1030-403b-b9f6-de633b04bf00 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:10.100369 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c11f40f4-1030-403b-b9f6-de633b04bf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:10.160755 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1225/2452] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:10.168979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909d0e5-cf87-4973-8a1c-09400c13c41e None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:10.169485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909d0e5-cf87-4973-8a1c-09400c13c41e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:10.169594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909d0e5-cf87-4973-8a1c-09400c13c41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:10.191807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-6909d0e5-cf87-4973-8a1c-09400c13c41e None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:10.193023 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6909d0e5-cf87-4973-8a1c-09400c13c41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:10.200848 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1228/2453] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:10.205173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f40f4-1030-403b-b9f6-de633b04bf00 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:10.215374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f40f4-1030-403b-b9f6-de633b04bf00 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:10.224822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1fab8-e90a-4b27-a8a5-fb174dfaade6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:10.225468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1fab8-e90a-4b27-a8a5-fb174dfaade6 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:10.225750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1fab8-e90a-4b27-a8a5-fb174dfaade6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:10.226046 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1fab8-e90a-4b27-a8a5-fb174dfaade6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:10.226744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1fab8-e90a-4b27-a8a5-fb174dfaade6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:48:10.227049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1fab8-e90a-4b27-a8a5-fb174dfaade6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:10.228664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1fab8-e90a-4b27-a8a5-fb174dfaade6 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:10.230531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:10.242429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1226/2454] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:48:10.250786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909d0e5-cf87-4973-8a1c-09400c13c41e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:10.260001 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909d0e5-cf87-4973-8a1c-09400c13c41e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:10.264778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32a317-5029-4d2b-8c6c-870fe39863c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:10.265391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32a317-5029-4d2b-8c6c-870fe39863c9 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:10.265670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32a317-5029-4d2b-8c6c-870fe39863c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:10.265942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32a317-5029-4d2b-8c6c-870fe39863c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:10.266633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32a317-5029-4d2b-8c6c-870fe39863c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:10.269319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32a317-5029-4d2b-8c6c-870fe39863c9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:10.332982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1229/2455] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:48:10 2022] POST /identity/v3/projects => generated 365 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:48:10.337105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1fab8-e90a-4b27-a8a5-fb174dfaade6 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:10.348375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1fab8-e90a-4b27-a8a5-fb174dfaade6 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:10.404283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3718d-c462-4fc8-ba01-aba31fe6a696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:10.404901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3718d-c462-4fc8-ba01-aba31fe6a696 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:10.405188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3718d-c462-4fc8-ba01-aba31fe6a696 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:10.405465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3718d-c462-4fc8-ba01-aba31fe6a696 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:10.406161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3718d-c462-4fc8-ba01-aba31fe6a696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:10.408078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3718d-c462-4fc8-ba01-aba31fe6a696 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:10.502560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1227/2456] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:48:10 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:48:10.507042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32a317-5029-4d2b-8c6c-870fe39863c9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:10.517603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32a317-5029-4d2b-8c6c-870fe39863c9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:10.524189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba2624-c7d6-44c8-b898-491fde62e174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:10.524986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba2624-c7d6-44c8-b898-491fde62e174 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:10.525237 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba2624-c7d6-44c8-b898-491fde62e174 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:10.525881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba2624-c7d6-44c8-b898-491fde62e174 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:10.526227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba2624-c7d6-44c8-b898-491fde62e174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:10.527990 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba2624-c7d6-44c8-b898-491fde62e174 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:10.540158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1230/2457] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:48:10.544449 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3718d-c462-4fc8-ba01-aba31fe6a696 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:10.554512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3718d-c462-4fc8-ba01-aba31fe6a696 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:10.559148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b47903-52fd-440e-a90e-2b51d562b710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:10.559756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b47903-52fd-440e-a90e-2b51d562b710 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:10.560041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b47903-52fd-440e-a90e-2b51d562b710 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:10.560316 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b47903-52fd-440e-a90e-2b51d562b710 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:10.561004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b47903-52fd-440e-a90e-2b51d562b710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:10.562639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b47903-52fd-440e-a90e-2b51d562b710 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:10.573558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1228/2458] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:48:10.577743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba2624-c7d6-44c8-b898-491fde62e174 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:10.588827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba2624-c7d6-44c8-b898-491fde62e174 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:10.593467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a077f59-810e-4db0-bbda-fd6754f6139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:10.594160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a077f59-810e-4db0-bbda-fd6754f6139d admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:10.594444 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a077f59-810e-4db0-bbda-fd6754f6139d admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:10.594721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a077f59-810e-4db0-bbda-fd6754f6139d admin admin] PATH_INFO: `/v3/projects/54dc0f66f0e54fd898a31aec00d4702e/users/717121eff13f4e868ac0b246aad6f063/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:10.609805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a077f59-810e-4db0-bbda-fd6754f6139d admin admin] RBAC: Authorizing `identity:create_grant(project_id=54dc0f66f0e54fd898a31aec00d4702e, user_id=717121eff13f4e868ac0b246aad6f063, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:10.612851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a077f59-810e-4db0-bbda-fd6754f6139d admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:10.668540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1231/2459] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:48:10 2022] PUT /identity/v3/projects/54dc0f66f0e54fd898a31aec00d4702e/users/717121eff13f4e868ac0b246aad6f063/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:10.680785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1276-1c32-4ccb-b4dc-3b64787b4b96 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:10.681075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1276-1c32-4ccb-b4dc-3b64787b4b96 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:10.681352 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1276-1c32-4ccb-b4dc-3b64787b4b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:10.709970 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-19df1276-1c32-4ccb-b4dc-3b64787b4b96 None None] MFA Rules not processed for user `717121eff13f4e868ac0b246aad6f063`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:10.711207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-19df1276-1c32-4ccb-b4dc-3b64787b4b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:10.775338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1229/2460] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:10.779939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a077f59-810e-4db0-bbda-fd6754f6139d admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:10.793655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:10.843012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a077f59-810e-4db0-bbda-fd6754f6139d admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:10.848355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04a58c-c3d7-46d4-bb30-e4880bdfcf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:10.848957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04a58c-c3d7-46d4-bb30-e4880bdfcf9c admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:10.849243 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04a58c-c3d7-46d4-bb30-e4880bdfcf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:10.849520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04a58c-c3d7-46d4-bb30-e4880bdfcf9c admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:10.850218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04a58c-c3d7-46d4-bb30-e4880bdfcf9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:10.852141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04a58c-c3d7-46d4-bb30-e4880bdfcf9c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:10.943700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1232/2461] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:48:10 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:48:10.948015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df1276-1c32-4ccb-b4dc-3b64787b4b96 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:10.959692 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df1276-1c32-4ccb-b4dc-3b64787b4b96 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:11.016398 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77297bbd-8cf8-4202-9b38-a3f258d2284c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:11.017022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77297bbd-8cf8-4202-9b38-a3f258d2284c admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:11.017307 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77297bbd-8cf8-4202-9b38-a3f258d2284c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:11.017583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77297bbd-8cf8-4202-9b38-a3f258d2284c admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:11.018771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77297bbd-8cf8-4202-9b38-a3f258d2284c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:11.020694 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77297bbd-8cf8-4202-9b38-a3f258d2284c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:11.029622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:11.153571 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1230/2462] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:48:10 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:48:11.157870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04a58c-c3d7-46d4-bb30-e4880bdfcf9c admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:11.169170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04a58c-c3d7-46d4-bb30-e4880bdfcf9c admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:11.173683 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8618b2-e4be-4afd-a4f1-c9493b5f0124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:11.174564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8618b2-e4be-4afd-a4f1-c9493b5f0124 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:11.174906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8618b2-e4be-4afd-a4f1-c9493b5f0124 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:11.175177 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8618b2-e4be-4afd-a4f1-c9493b5f0124 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:11.175862 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8618b2-e4be-4afd-a4f1-c9493b5f0124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:11.177450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8618b2-e4be-4afd-a4f1-c9493b5f0124 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:11.188399 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1233/2463] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:48:11.193558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77297bbd-8cf8-4202-9b38-a3f258d2284c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:11.203660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77297bbd-8cf8-4202-9b38-a3f258d2284c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:11.208957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911ab88-37d7-4031-b26a-865f593b8bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:11.209648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911ab88-37d7-4031-b26a-865f593b8bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:11.209932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911ab88-37d7-4031-b26a-865f593b8bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:11.210244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911ab88-37d7-4031-b26a-865f593b8bd9 admin admin] PATH_INFO: `/v3/projects/39b885a1ae4145998234508025a95872/users/c0412df795164b92a27258f2b7d4b7a0/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:11.225802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911ab88-37d7-4031-b26a-865f593b8bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b885a1ae4145998234508025a95872, user_id=c0412df795164b92a27258f2b7d4b7a0, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:11.228522 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911ab88-37d7-4031-b26a-865f593b8bd9 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:11.313930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1231/2464] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:48:11 2022] PUT /identity/v3/projects/39b885a1ae4145998234508025a95872/users/c0412df795164b92a27258f2b7d4b7a0/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:48:11.317915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8618b2-e4be-4afd-a4f1-c9493b5f0124 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:11.328061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8618b2-e4be-4afd-a4f1-c9493b5f0124 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:11.384304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8041cf-7f0e-464b-a6e3-c65f6b1ff36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:11.384925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8041cf-7f0e-464b-a6e3-c65f6b1ff36b admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:11.385207 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8041cf-7f0e-464b-a6e3-c65f6b1ff36b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:11.385484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8041cf-7f0e-464b-a6e3-c65f6b1ff36b admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:11.386207 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8041cf-7f0e-464b-a6e3-c65f6b1ff36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:11.387794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8041cf-7f0e-464b-a6e3-c65f6b1ff36b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:11.398242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1234/2465] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:48:11.402664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911ab88-37d7-4031-b26a-865f593b8bd9 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:11.413214 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911ab88-37d7-4031-b26a-865f593b8bd9 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:11.423463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ceaf8c-58e9-49eb-8efa-8f18fc0aa77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:11.424191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ceaf8c-58e9-49eb-8efa-8f18fc0aa77d admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:11.424483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ceaf8c-58e9-49eb-8efa-8f18fc0aa77d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:11.424759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ceaf8c-58e9-49eb-8efa-8f18fc0aa77d admin admin] PATH_INFO: `/v3/projects/39b885a1ae4145998234508025a95872/users/c0412df795164b92a27258f2b7d4b7a0/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:11.426869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ceaf8c-58e9-49eb-8efa-8f18fc0aa77d admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b885a1ae4145998234508025a95872, user_id=c0412df795164b92a27258f2b7d4b7a0, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:11.429543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ceaf8c-58e9-49eb-8efa-8f18fc0aa77d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:11.441418 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1232/2466] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:48:11 2022] PUT /identity/v3/projects/39b885a1ae4145998234508025a95872/users/c0412df795164b92a27258f2b7d4b7a0/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:48:11.451801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16947bd-a8e2-4f81-b208-62c2b00ff1e3 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:11.452089 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16947bd-a8e2-4f81-b208-62c2b00ff1e3 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:11.452365 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16947bd-a8e2-4f81-b208-62c2b00ff1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:11.481816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-b16947bd-a8e2-4f81-b208-62c2b00ff1e3 None None] MFA Rules not processed for user `c0412df795164b92a27258f2b7d4b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:11.483154 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b16947bd-a8e2-4f81-b208-62c2b00ff1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:11.553733 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1235/2467] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:11.565766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ceaf8c-58e9-49eb-8efa-8f18fc0aa77d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:11.567425 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-87ceaf8c-58e9-49eb-8efa-8f18fc0aa77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:11.624393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ceaf8c-58e9-49eb-8efa-8f18fc0aa77d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:11.628967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b480f8-1afd-4b47-a589-d1e38dd0cd44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:11.629589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b480f8-1afd-4b47-a589-d1e38dd0cd44 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:11.629878 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b480f8-1afd-4b47-a589-d1e38dd0cd44 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:11.630195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b480f8-1afd-4b47-a589-d1e38dd0cd44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:11.639428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b480f8-1afd-4b47-a589-d1e38dd0cd44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:11.640867 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b480f8-1afd-4b47-a589-d1e38dd0cd44 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:11.650965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1233/2468] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:11.871000 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16947bd-a8e2-4f81-b208-62c2b00ff1e3 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:11.881948 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16947bd-a8e2-4f81-b208-62c2b00ff1e3 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:11.885534 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce91e01-702b-4583-9dc9-f80f99ee682f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:11.886191 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce91e01-702b-4583-9dc9-f80f99ee682f service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:11.886473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce91e01-702b-4583-9dc9-f80f99ee682f service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:11.886751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce91e01-702b-4583-9dc9-f80f99ee682f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:11.892095 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:11.946697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce91e01-702b-4583-9dc9-f80f99ee682f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:11.949010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce91e01-702b-4583-9dc9-f80f99ee682f service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:11.959119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1236/2469] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:21.646683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:INSERT=4,keystone:SELECT=43 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:21.954341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=24 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:24.560137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b480f8-1afd-4b47-a589-d1e38dd0cd44 service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:24.662112 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b480f8-1afd-4b47-a589-d1e38dd0cd44 service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:24.666680 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:24.667314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:24.667599 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:24.667875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] PATH_INFO: `/v3/users/717121eff13f4e868ac0b246aad6f063` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:24.669067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] RBAC: Authorizing `identity:delete_user(user_id=717121eff13f4e868ac0b246aad6f063)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:24.671020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:24.823469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '717121eff13f4e868ac0b246aad6f063'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:24.879090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '717121eff13f4e868ac0b246aad6f063'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:24.879708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '717121eff13f4e868ac0b246aad6f063'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:24.891937 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '717121eff13f4e868ac0b246aad6f063'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:24.899811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '717121eff13f4e868ac0b246aad6f063'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:24.905670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1234/2470] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:48:24 2022] DELETE /identity/v3/users/717121eff13f4e868ac0b246aad6f063 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:24.910410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c96cc6-b810-4824-bf94-b9797c379f6d None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:24.910906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c96cc6-b810-4824-bf94-b9797c379f6d None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:24.911020 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c96cc6-b810-4824-bf94-b9797c379f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:24.960566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f6c96cc6-b810-4824-bf94-b9797c379f6d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:24.961767 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f6c96cc6-b810-4824-bf94-b9797c379f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:25.023667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1237/2471] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:25.033409 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:25.034836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:25.075585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:25.107223 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ebda0-5a42-4f73-88d5-49b31dc8700c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:25.110819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29ca63-4b22-417e-a494-a81ec48ab27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:25.111498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29ca63-4b22-417e-a494-a81ec48ab27d service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:25.111846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29ca63-4b22-417e-a494-a81ec48ab27d service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:25.112118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29ca63-4b22-417e-a494-a81ec48ab27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:25.121545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29ca63-4b22-417e-a494-a81ec48ab27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:25.123268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29ca63-4b22-417e-a494-a81ec48ab27d service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:25.133073 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1235/2472] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:25.561033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c96cc6-b810-4824-bf94-b9797c379f6d None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:25.562613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f6c96cc6-b810-4824-bf94-b9797c379f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:25.576687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c96cc6-b810-4824-bf94-b9797c379f6d None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:25.581676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:25.582577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:25.582860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:25.583138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] PATH_INFO: `/v3/projects/54dc0f66f0e54fd898a31aec00d4702e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:25.584342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54dc0f66f0e54fd898a31aec00d4702e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:25.586430 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:25.690994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54dc0f66f0e54fd898a31aec00d4702e', 'request_id': 'req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:25.741410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54dc0f66f0e54fd898a31aec00d4702e', 'request_id': 'req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:25.747431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54dc0f66f0e54fd898a31aec00d4702e', 'request_id': 'req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:25.758798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] The token cache is being invalidate because project 54dc0f66f0e54fd898a31aec00d4702e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:48:25.759083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:25.759713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:25.761257 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1238/2473] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:48:25 2022] DELETE /identity/v3/projects/54dc0f66f0e54fd898a31aec00d4702e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:25.765382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29ca63-4b22-417e-a494-a81ec48ab27d service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:25.775689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29ca63-4b22-417e-a494-a81ec48ab27d service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:25.831382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcea6a-30d5-4cc5-bff8-9cef374cb129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:25.832021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcea6a-30d5-4cc5-bff8-9cef374cb129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:25.832305 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcea6a-30d5-4cc5-bff8-9cef374cb129 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:25.832582 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcea6a-30d5-4cc5-bff8-9cef374cb129 admin admin] PATH_INFO: `/v3/users/c0412df795164b92a27258f2b7d4b7a0` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:25.833785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcea6a-30d5-4cc5-bff8-9cef374cb129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0412df795164b92a27258f2b7d4b7a0)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:25.835743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcea6a-30d5-4cc5-bff8-9cef374cb129 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:25.989220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ccdcea6a-30d5-4cc5-bff8-9cef374cb129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0412df795164b92a27258f2b7d4b7a0'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:26.049871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ccdcea6a-30d5-4cc5-bff8-9cef374cb129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0412df795164b92a27258f2b7d4b7a0'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:26.050668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ccdcea6a-30d5-4cc5-bff8-9cef374cb129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0412df795164b92a27258f2b7d4b7a0'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:26.062846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ccdcea6a-30d5-4cc5-bff8-9cef374cb129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0412df795164b92a27258f2b7d4b7a0'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:26.069619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ccdcea6a-30d5-4cc5-bff8-9cef374cb129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0412df795164b92a27258f2b7d4b7a0'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:26.075583 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1236/2474] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:48:25 2022] DELETE /identity/v3/users/c0412df795164b92a27258f2b7d4b7a0 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:26.486395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:26.487819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:26.501577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f86d9-7a25-4fd2-b8e5-e60175dcc6e1 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:26.560975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:26.567737 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:26.568377 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:26.568656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:26.568997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] PATH_INFO: `/v3/projects/39b885a1ae4145998234508025a95872` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:26.570171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39b885a1ae4145998234508025a95872)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:26.572122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:26.712612 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39b885a1ae4145998234508025a95872', 'request_id': 'req-cd169a51-6e1f-4076-9f51-0d49355be6c1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:26.752932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39b885a1ae4145998234508025a95872', 'request_id': 'req-cd169a51-6e1f-4076-9f51-0d49355be6c1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:26.758798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39b885a1ae4145998234508025a95872', 'request_id': 'req-cd169a51-6e1f-4076-9f51-0d49355be6c1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:26.772116 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] The token cache is being invalidate because project 39b885a1ae4145998234508025a95872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:48:26.772407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:26.773104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-cd169a51-6e1f-4076-9f51-0d49355be6c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:26.774666 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1239/2475] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:48:26 2022] DELETE /identity/v3/projects/39b885a1ae4145998234508025a95872 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:26.780965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d4d4d-4eb9-4d1d-8c09-ad5bf851e676 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:26.781465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d4d4d-4eb9-4d1d-8c09-ad5bf851e676 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:26.781465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d4d4d-4eb9-4d1d-8c09-ad5bf851e676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:26.835300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-914d4d4d-4eb9-4d1d-8c09-ad5bf851e676 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:26.837140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-914d4d4d-4eb9-4d1d-8c09-ad5bf851e676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:26.851093 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:26.898647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1237/2476] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:26.914356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755f798-727a-4e85-8fd6-09b1a1fe21ff None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:26.914637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755f798-727a-4e85-8fd6-09b1a1fe21ff None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:26.914935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755f798-727a-4e85-8fd6-09b1a1fe21ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:26.935624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1755f798-727a-4e85-8fd6-09b1a1fe21ff None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:26.939348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1755f798-727a-4e85-8fd6-09b1a1fe21ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:26.945120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1240/2477] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:26.949414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d4d4d-4eb9-4d1d-8c09-ad5bf851e676 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:26.960888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d4d4d-4eb9-4d1d-8c09-ad5bf851e676 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:26.965730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0a9c0-114a-45bf-a78c-3b112caaaa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:26.966391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0a9c0-114a-45bf-a78c-3b112caaaa98 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:26.966709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0a9c0-114a-45bf-a78c-3b112caaaa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:26.966987 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0a9c0-114a-45bf-a78c-3b112caaaa98 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:26.967736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0a9c0-114a-45bf-a78c-3b112caaaa98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:48:26.968050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0a9c0-114a-45bf-a78c-3b112caaaa98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:26.969706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0a9c0-114a-45bf-a78c-3b112caaaa98 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:26.982660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1238/2478] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:48:26.987320 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755f798-727a-4e85-8fd6-09b1a1fe21ff None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:26.996465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755f798-727a-4e85-8fd6-09b1a1fe21ff None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:27.001562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704750f-290f-4031-926b-a99cee2a520a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:27.002600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704750f-290f-4031-926b-a99cee2a520a admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:27.002883 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704750f-290f-4031-926b-a99cee2a520a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:27.003153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704750f-290f-4031-926b-a99cee2a520a admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:27.003818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704750f-290f-4031-926b-a99cee2a520a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:27.005701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704750f-290f-4031-926b-a99cee2a520a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:27.091692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1241/2479] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:48:26 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:48:27.096298 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0a9c0-114a-45bf-a78c-3b112caaaa98 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:27.106468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0a9c0-114a-45bf-a78c-3b112caaaa98 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:27.163806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb8706-0a12-4766-9b78-3509f752c6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:27.164423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb8706-0a12-4766-9b78-3509f752c6cf admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:27.164707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb8706-0a12-4766-9b78-3509f752c6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:27.164988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb8706-0a12-4766-9b78-3509f752c6cf admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:27.165648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb8706-0a12-4766-9b78-3509f752c6cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:27.167615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb8706-0a12-4766-9b78-3509f752c6cf admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:27.285797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1239/2480] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:48:27 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:48:27.289852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704750f-290f-4031-926b-a99cee2a520a admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:27.300322 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704750f-290f-4031-926b-a99cee2a520a admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:27.304996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0f8b1-e94a-4dad-8ec4-b3115abd46e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:27.305592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0f8b1-e94a-4dad-8ec4-b3115abd46e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:27.305871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0f8b1-e94a-4dad-8ec4-b3115abd46e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:27.306182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0f8b1-e94a-4dad-8ec4-b3115abd46e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:27.306867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0f8b1-e94a-4dad-8ec4-b3115abd46e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:27.308441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0f8b1-e94a-4dad-8ec4-b3115abd46e1 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:27.319087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1242/2481] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:48:27.323129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb8706-0a12-4766-9b78-3509f752c6cf admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:27.333825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb8706-0a12-4766-9b78-3509f752c6cf admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:27.338511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3f238-0b15-4379-9629-09fd15b6189c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:27.339610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3f238-0b15-4379-9629-09fd15b6189c admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:27.339610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3f238-0b15-4379-9629-09fd15b6189c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:27.339803 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3f238-0b15-4379-9629-09fd15b6189c admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:27.340381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3f238-0b15-4379-9629-09fd15b6189c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:27.342134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3f238-0b15-4379-9629-09fd15b6189c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:27.352597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1240/2482] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:48:27.356633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0f8b1-e94a-4dad-8ec4-b3115abd46e1 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:27.366590 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0f8b1-e94a-4dad-8ec4-b3115abd46e1 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:27.371100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab72a1-2083-4584-8fff-118ab19631d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:27.371743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab72a1-2083-4584-8fff-118ab19631d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:27.372274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab72a1-2083-4584-8fff-118ab19631d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:27.374093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab72a1-2083-4584-8fff-118ab19631d4 admin admin] PATH_INFO: `/v3/projects/f5a51d538c6c4300b6f8902791ca78fa/users/9fafaa54f142406cafb8e85d3d2b3309/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:27.390161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab72a1-2083-4584-8fff-118ab19631d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a51d538c6c4300b6f8902791ca78fa, user_id=9fafaa54f142406cafb8e85d3d2b3309, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:27.393772 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab72a1-2083-4584-8fff-118ab19631d4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:27.462306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1243/2483] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:48:27 2022] PUT /identity/v3/projects/f5a51d538c6c4300b6f8902791ca78fa/users/9fafaa54f142406cafb8e85d3d2b3309/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:27.474269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dc148-a1a9-4e81-89ce-00b5a977d9d1 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:27.474562 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dc148-a1a9-4e81-89ce-00b5a977d9d1 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:27.474845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dc148-a1a9-4e81-89ce-00b5a977d9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:27.503660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-4c1dc148-a1a9-4e81-89ce-00b5a977d9d1 None None] MFA Rules not processed for user `9fafaa54f142406cafb8e85d3d2b3309`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:27.504869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4c1dc148-a1a9-4e81-89ce-00b5a977d9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:27.569917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1241/2484] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:27.574385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab72a1-2083-4584-8fff-118ab19631d4 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:27.635137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab72a1-2083-4584-8fff-118ab19631d4 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:27.639735 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7a419-e5dd-4c88-80e8-98c168236433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:27.640343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7a419-e5dd-4c88-80e8-98c168236433 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:27.640627 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7a419-e5dd-4c88-80e8-98c168236433 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:27.640900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7a419-e5dd-4c88-80e8-98c168236433 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:27.641570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7a419-e5dd-4c88-80e8-98c168236433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:27.643731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7a419-e5dd-4c88-80e8-98c168236433 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:27.749063 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1244/2485] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:48:27 2022] POST /identity/v3/projects => generated 381 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:48:27.754027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dc148-a1a9-4e81-89ce-00b5a977d9d1 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:27.765099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dc148-a1a9-4e81-89ce-00b5a977d9d1 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:27.776881 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:27.825761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c30937-92a2-47d7-a14d-0e372546f84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:27.826489 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c30937-92a2-47d7-a14d-0e372546f84d admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:27.826693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c30937-92a2-47d7-a14d-0e372546f84d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:27.826990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c30937-92a2-47d7-a14d-0e372546f84d admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:27.827687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c30937-92a2-47d7-a14d-0e372546f84d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:27.829655 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c30937-92a2-47d7-a14d-0e372546f84d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:27.954078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1242/2486] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:48:27 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:48:27.958418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7a419-e5dd-4c88-80e8-98c168236433 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:27.968993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7a419-e5dd-4c88-80e8-98c168236433 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:27.974948 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d370248-4c5c-4fc8-b099-ab821b663a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:27.975594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370248-4c5c-4fc8-b099-ab821b663a61 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:27.975874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370248-4c5c-4fc8-b099-ab821b663a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:27.976147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370248-4c5c-4fc8-b099-ab821b663a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:27.976844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d370248-4c5c-4fc8-b099-ab821b663a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:27.978543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d370248-4c5c-4fc8-b099-ab821b663a61 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:27.980978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:27.991114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1245/2487] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:48:28.002090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c30937-92a2-47d7-a14d-0e372546f84d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:28.012662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c30937-92a2-47d7-a14d-0e372546f84d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:28.017651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8c111-55a5-4b8d-b410-60d453b3bb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:28.018426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8c111-55a5-4b8d-b410-60d453b3bb30 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:28.018710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8c111-55a5-4b8d-b410-60d453b3bb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:28.019031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8c111-55a5-4b8d-b410-60d453b3bb30 admin admin] PATH_INFO: `/v3/projects/b0f38b4384cd40028b2e74ecb9fb558f/users/a620a7ff74724043b329ed1c52c7e4c8/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:28.038135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8c111-55a5-4b8d-b410-60d453b3bb30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f38b4384cd40028b2e74ecb9fb558f, user_id=a620a7ff74724043b329ed1c52c7e4c8, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:28.040827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8c111-55a5-4b8d-b410-60d453b3bb30 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:28.153525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1243/2488] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:48:28 2022] PUT /identity/v3/projects/b0f38b4384cd40028b2e74ecb9fb558f/users/a620a7ff74724043b329ed1c52c7e4c8/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:28.157654 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d370248-4c5c-4fc8-b099-ab821b663a61 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:28.168827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d370248-4c5c-4fc8-b099-ab821b663a61 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:28.227146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dac0a-b4dc-41ca-8e0b-06610ae8cd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:28.227678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dac0a-b4dc-41ca-8e0b-06610ae8cd18 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:28.227958 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dac0a-b4dc-41ca-8e0b-06610ae8cd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:28.228251 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dac0a-b4dc-41ca-8e0b-06610ae8cd18 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:28.228983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dac0a-b4dc-41ca-8e0b-06610ae8cd18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:28.230774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dac0a-b4dc-41ca-8e0b-06610ae8cd18 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:28.242101 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1246/2489] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:48:28.246485 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8c111-55a5-4b8d-b410-60d453b3bb30 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:28.257569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8c111-55a5-4b8d-b410-60d453b3bb30 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:28.262281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5a509-4f3e-43a2-a544-7a270207e98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:28.262935 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5a509-4f3e-43a2-a544-7a270207e98c admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:28.263215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5a509-4f3e-43a2-a544-7a270207e98c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:28.263487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5a509-4f3e-43a2-a544-7a270207e98c admin admin] PATH_INFO: `/v3/projects/b0f38b4384cd40028b2e74ecb9fb558f/users/a620a7ff74724043b329ed1c52c7e4c8/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:28.265480 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5a509-4f3e-43a2-a544-7a270207e98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f38b4384cd40028b2e74ecb9fb558f, user_id=a620a7ff74724043b329ed1c52c7e4c8, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:28.268735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5a509-4f3e-43a2-a544-7a270207e98c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:28.278825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1244/2490] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:48:28 2022] PUT /identity/v3/projects/b0f38b4384cd40028b2e74ecb9fb558f/users/a620a7ff74724043b329ed1c52c7e4c8/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:28.290276 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76174a82-d320-4215-9054-bb5ebdc14b30 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:28.290563 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76174a82-d320-4215-9054-bb5ebdc14b30 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:28.291015 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76174a82-d320-4215-9054-bb5ebdc14b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:28.318669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-76174a82-d320-4215-9054-bb5ebdc14b30 None None] MFA Rules not processed for user `a620a7ff74724043b329ed1c52c7e4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:28.319897 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-76174a82-d320-4215-9054-bb5ebdc14b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:28.388570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1247/2491] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:28.401614 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5a509-4f3e-43a2-a544-7a270207e98c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:28.403434 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-02e5a509-4f3e-43a2-a544-7a270207e98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:28.464385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5a509-4f3e-43a2-a544-7a270207e98c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:28.468055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567afd6f-6097-474d-a8f0-625962fa4641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:28.468649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567afd6f-6097-474d-a8f0-625962fa4641 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:28.468963 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567afd6f-6097-474d-a8f0-625962fa4641 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:28.469902 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567afd6f-6097-474d-a8f0-625962fa4641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:28.479488 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567afd6f-6097-474d-a8f0-625962fa4641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:28.480852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567afd6f-6097-474d-a8f0-625962fa4641 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:28.490900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1245/2492] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:28.637611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76174a82-d320-4215-9054-bb5ebdc14b30 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:28.648525 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76174a82-d320-4215-9054-bb5ebdc14b30 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:28.652180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a74ef-841d-4260-a4bb-29602f9697cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:28.652798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a74ef-841d-4260-a4bb-29602f9697cf service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:28.653081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a74ef-841d-4260-a4bb-29602f9697cf service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:28.653358 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a74ef-841d-4260-a4bb-29602f9697cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:28.714286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a74ef-841d-4260-a4bb-29602f9697cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:28.715708 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a74ef-841d-4260-a4bb-29602f9697cf service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:28.725125 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1248/2493] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:29.244934 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567afd6f-6097-474d-a8f0-625962fa4641 service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:29.312169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567afd6f-6097-474d-a8f0-625962fa4641 service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:29.317551 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:29.318805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:29.319090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:29.319362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] PATH_INFO: `/v3/users/9fafaa54f142406cafb8e85d3d2b3309` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:29.321056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fafaa54f142406cafb8e85d3d2b3309)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:29.323106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:29.338121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:29.492994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fafaa54f142406cafb8e85d3d2b3309'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:29.553519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fafaa54f142406cafb8e85d3d2b3309'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:29.554173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fafaa54f142406cafb8e85d3d2b3309'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:29.566941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fafaa54f142406cafb8e85d3d2b3309'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:29.573950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fafaa54f142406cafb8e85d3d2b3309'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:29.580596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1246/2494] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:48:29 2022] DELETE /identity/v3/users/9fafaa54f142406cafb8e85d3d2b3309 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:29.585270 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67a45b-6053-4433-9adb-3fda37b16c24 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:29.585562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67a45b-6053-4433-9adb-3fda37b16c24 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:29.585843 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67a45b-6053-4433-9adb-3fda37b16c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:29.606785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bb67a45b-6053-4433-9adb-3fda37b16c24 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:29.608047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bb67a45b-6053-4433-9adb-3fda37b16c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:29.620178 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:29.667740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1249/2495] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:29.678189 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:29.679833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:29.753011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eff6f-3019-4107-9162-6acc8a0b12ff admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:29.756872 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa8132-0b1f-4dab-bad1-f4f261156d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:29.757511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa8132-0b1f-4dab-bad1-f4f261156d61 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:29.757792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa8132-0b1f-4dab-bad1-f4f261156d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:29.758090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa8132-0b1f-4dab-bad1-f4f261156d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:29.768415 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa8132-0b1f-4dab-bad1-f4f261156d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:29.769876 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa8132-0b1f-4dab-bad1-f4f261156d61 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:29.779764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1247/2496] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:30.190728 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67a45b-6053-4433-9adb-3fda37b16c24 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:30.192331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bb67a45b-6053-4433-9adb-3fda37b16c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:30.211122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67a45b-6053-4433-9adb-3fda37b16c24 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:30.211122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:30.211122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:30.211122 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:30.211501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] PATH_INFO: `/v3/projects/f5a51d538c6c4300b6f8902791ca78fa` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:30.212840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a51d538c6c4300b6f8902791ca78fa)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:30.214458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:30.318528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a51d538c6c4300b6f8902791ca78fa', 'request_id': 'req-3b2604df-329b-4bfc-8932-b99d3ff09156'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:30.364240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a51d538c6c4300b6f8902791ca78fa', 'request_id': 'req-3b2604df-329b-4bfc-8932-b99d3ff09156'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:30.370342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a51d538c6c4300b6f8902791ca78fa', 'request_id': 'req-3b2604df-329b-4bfc-8932-b99d3ff09156'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:30.381925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] The token cache is being invalidate because project f5a51d538c6c4300b6f8902791ca78fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:48:30.382844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:30.382844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:30.384418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1250/2497] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:48:30 2022] DELETE /identity/v3/projects/f5a51d538c6c4300b6f8902791ca78fa => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:48:30.388475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa8132-0b1f-4dab-bad1-f4f261156d61 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:30.398744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa8132-0b1f-4dab-bad1-f4f261156d61 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:30.456277 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978de710-baae-4b65-88fe-9bbf4ad7355e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:30.456838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978de710-baae-4b65-88fe-9bbf4ad7355e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:30.457123 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978de710-baae-4b65-88fe-9bbf4ad7355e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:30.457440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978de710-baae-4b65-88fe-9bbf4ad7355e admin admin] PATH_INFO: `/v3/users/a620a7ff74724043b329ed1c52c7e4c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:30.458688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978de710-baae-4b65-88fe-9bbf4ad7355e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a620a7ff74724043b329ed1c52c7e4c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:30.460668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978de710-baae-4b65-88fe-9bbf4ad7355e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:30.627186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:30.636115 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-978de710-baae-4b65-88fe-9bbf4ad7355e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a620a7ff74724043b329ed1c52c7e4c8'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:30.730420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-978de710-baae-4b65-88fe-9bbf4ad7355e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a620a7ff74724043b329ed1c52c7e4c8'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:30.731103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-978de710-baae-4b65-88fe-9bbf4ad7355e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a620a7ff74724043b329ed1c52c7e4c8'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:30.745554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-978de710-baae-4b65-88fe-9bbf4ad7355e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a620a7ff74724043b329ed1c52c7e4c8'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:30.753858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-978de710-baae-4b65-88fe-9bbf4ad7355e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a620a7ff74724043b329ed1c52c7e4c8'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:30.759985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1248/2498] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:48:30 2022] DELETE /identity/v3/users/a620a7ff74724043b329ed1c52c7e4c8 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:31.183355 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:31.184814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:31.200266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2604df-329b-4bfc-8932-b99d3ff09156 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:31.261601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:31.262187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:31.262471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:31.262747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] PATH_INFO: `/v3/projects/b0f38b4384cd40028b2e74ecb9fb558f` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:31.263915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f38b4384cd40028b2e74ecb9fb558f)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:31.265821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:31.367808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:31.371428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f38b4384cd40028b2e74ecb9fb558f', 'request_id': 'req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:31.450222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f38b4384cd40028b2e74ecb9fb558f', 'request_id': 'req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:31.456898 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f38b4384cd40028b2e74ecb9fb558f', 'request_id': 'req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:31.469560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] The token cache is being invalidate because project b0f38b4384cd40028b2e74ecb9fb558f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:48:31.469869 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:31.471205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f2a6f5e0-7dc6-443f-8154-52b7c97376cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:31.472850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1251/2499] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:48:31 2022] DELETE /identity/v3/projects/b0f38b4384cd40028b2e74ecb9fb558f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:31.477719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974af17-a093-4f65-bdaf-b5fbb99a6739 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:31.488436 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974af17-a093-4f65-bdaf-b5fbb99a6739 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:31.488436 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974af17-a093-4f65-bdaf-b5fbb99a6739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:31.498567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-a974af17-a093-4f65-bdaf-b5fbb99a6739 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:31.499790 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a974af17-a093-4f65-bdaf-b5fbb99a6739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:31.565705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1249/2500] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:31.574958 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada9878-f0e1-4300-b776-4010a81408e5 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:31.575245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada9878-f0e1-4300-b776-4010a81408e5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:31.575520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada9878-f0e1-4300-b776-4010a81408e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:31.598982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-fada9878-f0e1-4300-b776-4010a81408e5 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:31.600209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-fada9878-f0e1-4300-b776-4010a81408e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:31.608220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1252/2501] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:31.613329 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974af17-a093-4f65-bdaf-b5fbb99a6739 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:31.625641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974af17-a093-4f65-bdaf-b5fbb99a6739 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:31.630836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc34669-a1f8-4b39-b99b-0151a0266870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:31.631455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc34669-a1f8-4b39-b99b-0151a0266870 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:31.631741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc34669-a1f8-4b39-b99b-0151a0266870 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:31.632014 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc34669-a1f8-4b39-b99b-0151a0266870 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:31.632713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc34669-a1f8-4b39-b99b-0151a0266870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:48:31.633023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc34669-a1f8-4b39-b99b-0151a0266870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:31.634672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc34669-a1f8-4b39-b99b-0151a0266870 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:31.647348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1250/2502] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:48:31.651779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada9878-f0e1-4300-b776-4010a81408e5 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:31.661392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada9878-f0e1-4300-b776-4010a81408e5 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:31.666126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fdcd9-ea90-4d5f-ad1f-eb13e418a81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:31.666731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fdcd9-ea90-4d5f-ad1f-eb13e418a81f admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:31.667013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fdcd9-ea90-4d5f-ad1f-eb13e418a81f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:31.667287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fdcd9-ea90-4d5f-ad1f-eb13e418a81f admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:31.667944 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fdcd9-ea90-4d5f-ad1f-eb13e418a81f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:31.669847 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fdcd9-ea90-4d5f-ad1f-eb13e418a81f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:31.801418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1253/2503] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:48:31 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:48:31.805779 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc34669-a1f8-4b39-b99b-0151a0266870 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:31.817110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc34669-a1f8-4b39-b99b-0151a0266870 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:31.879454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5bbc2-3c7b-4349-acc7-baaabe34c20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:31.880064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5bbc2-3c7b-4349-acc7-baaabe34c20a admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:31.880353 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5bbc2-3c7b-4349-acc7-baaabe34c20a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:31.880632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5bbc2-3c7b-4349-acc7-baaabe34c20a admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:31.881317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5bbc2-3c7b-4349-acc7-baaabe34c20a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:31.883297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5bbc2-3c7b-4349-acc7-baaabe34c20a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:31.988362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1251/2504] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:48:31 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:48:31.992538 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fdcd9-ea90-4d5f-ad1f-eb13e418a81f admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:32.004376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fdcd9-ea90-4d5f-ad1f-eb13e418a81f admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:32.009036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc9dcb-a96e-40b5-9af6-779fdf5e4023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:32.014083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc9dcb-a96e-40b5-9af6-779fdf5e4023 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:32.014083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc9dcb-a96e-40b5-9af6-779fdf5e4023 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:32.014083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc9dcb-a96e-40b5-9af6-779fdf5e4023 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:32.014083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc9dcb-a96e-40b5-9af6-779fdf5e4023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:32.014083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc9dcb-a96e-40b5-9af6-779fdf5e4023 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:32.023086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1254/2505] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:48:32.027364 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5bbc2-3c7b-4349-acc7-baaabe34c20a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:32.038301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5bbc2-3c7b-4349-acc7-baaabe34c20a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:32.043003 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89541ac4-41c0-484c-a466-158fe52dcd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:32.043670 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89541ac4-41c0-484c-a466-158fe52dcd72 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:32.043953 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89541ac4-41c0-484c-a466-158fe52dcd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:32.044229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89541ac4-41c0-484c-a466-158fe52dcd72 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:32.044914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89541ac4-41c0-484c-a466-158fe52dcd72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:32.046550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89541ac4-41c0-484c-a466-158fe52dcd72 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:32.058651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1252/2506] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:48:32.062567 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc9dcb-a96e-40b5-9af6-779fdf5e4023 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:32.074661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc9dcb-a96e-40b5-9af6-779fdf5e4023 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:32.079189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee33448-3871-474f-9c27-feff7acef879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:32.079850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33448-3871-474f-9c27-feff7acef879 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:32.080331 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33448-3871-474f-9c27-feff7acef879 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:32.080770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33448-3871-474f-9c27-feff7acef879 admin admin] PATH_INFO: `/v3/projects/9cbf429d461042238296518f0884c689/users/e4c84ab662d74f249e29644d6bd28fcd/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:32.096472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee33448-3871-474f-9c27-feff7acef879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cbf429d461042238296518f0884c689, user_id=e4c84ab662d74f249e29644d6bd28fcd, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:32.099172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee33448-3871-474f-9c27-feff7acef879 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:32.165827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1255/2507] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:48:32 2022] PUT /identity/v3/projects/9cbf429d461042238296518f0884c689/users/e4c84ab662d74f249e29644d6bd28fcd/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:32.171688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31118-9fcb-4a36-bd9c-fee72bd63e1f None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:32.172023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31118-9fcb-4a36-bd9c-fee72bd63e1f None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:32.172379 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31118-9fcb-4a36-bd9c-fee72bd63e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:32.194309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-f4b31118-9fcb-4a36-bd9c-fee72bd63e1f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:32.195545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f4b31118-9fcb-4a36-bd9c-fee72bd63e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:32.208855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:32.256959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1253/2508] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:32.267084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee33448-3871-474f-9c27-feff7acef879 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:32.268940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-dee33448-3871-474f-9c27-feff7acef879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:32.336754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee33448-3871-474f-9c27-feff7acef879 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:32.340621 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506df636-6751-4a73-8e8f-8fba12739ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:32.341240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506df636-6751-4a73-8e8f-8fba12739ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:32.341521 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506df636-6751-4a73-8e8f-8fba12739ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:32.341793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506df636-6751-4a73-8e8f-8fba12739ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:32.352136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506df636-6751-4a73-8e8f-8fba12739ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:32.353627 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506df636-6751-4a73-8e8f-8fba12739ed8 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:32.363222 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1256/2509] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:33.595108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a5df5-52c8-4934-98f4-d70b78a8eaa5 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:33.595696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a5df5-52c8-4934-98f4-d70b78a8eaa5 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:33.595696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a5df5-52c8-4934-98f4-d70b78a8eaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:33.626773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-e45a5df5-52c8-4934-98f4-d70b78a8eaa5 None None] MFA Rules not processed for user `e4c84ab662d74f249e29644d6bd28fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:33.628061 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e45a5df5-52c8-4934-98f4-d70b78a8eaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:33.700370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1254/2510] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:33.704930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506df636-6751-4a73-8e8f-8fba12739ed8 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:33.715081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506df636-6751-4a73-8e8f-8fba12739ed8 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:33.719576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea7879-556e-4b20-b938-54d09d7391b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:33.720173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea7879-556e-4b20-b938-54d09d7391b6 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:33.720453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea7879-556e-4b20-b938-54d09d7391b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:33.720803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea7879-556e-4b20-b938-54d09d7391b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:33.722068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea7879-556e-4b20-b938-54d09d7391b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:33.724079 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea7879-556e-4b20-b938-54d09d7391b6 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:33.823998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1257/2511] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:48:33 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:48:33.827805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a5df5-52c8-4934-98f4-d70b78a8eaa5 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:33.839327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a5df5-52c8-4934-98f4-d70b78a8eaa5 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:33.903519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad9b81-b84c-4c74-a757-2e8827009802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:33.904271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad9b81-b84c-4c74-a757-2e8827009802 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:33.904660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad9b81-b84c-4c74-a757-2e8827009802 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:33.905076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad9b81-b84c-4c74-a757-2e8827009802 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:33.905846 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad9b81-b84c-4c74-a757-2e8827009802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:33.907971 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad9b81-b84c-4c74-a757-2e8827009802 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:34.017146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1255/2512] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:48:33 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:48:34.022195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea7879-556e-4b20-b938-54d09d7391b6 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:34.032429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea7879-556e-4b20-b938-54d09d7391b6 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:34.038135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53294b1-ea36-4745-9c56-37807d79d448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:34.038765 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53294b1-ea36-4745-9c56-37807d79d448 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:34.039047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53294b1-ea36-4745-9c56-37807d79d448 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:34.039321 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53294b1-ea36-4745-9c56-37807d79d448 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:34.040019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53294b1-ea36-4745-9c56-37807d79d448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:34.041635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53294b1-ea36-4745-9c56-37807d79d448 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:34.053839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1258/2513] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:48:34.059463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad9b81-b84c-4c74-a757-2e8827009802 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:34.073907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad9b81-b84c-4c74-a757-2e8827009802 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:34.078828 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c10efb-91dc-431a-9363-f1f182519420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:34.079445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c10efb-91dc-431a-9363-f1f182519420 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:34.079731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c10efb-91dc-431a-9363-f1f182519420 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:34.080697 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c10efb-91dc-431a-9363-f1f182519420 admin admin] PATH_INFO: `/v3/projects/2faa26a1fcd54070a97ecce3a6a1d0fb/users/d5e8d59842a1403f95f3f31cc5a6b520/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:34.098354 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c10efb-91dc-431a-9363-f1f182519420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2faa26a1fcd54070a97ecce3a6a1d0fb, user_id=d5e8d59842a1403f95f3f31cc5a6b520, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:34.101031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c10efb-91dc-431a-9363-f1f182519420 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:34.182097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1256/2514] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:48:34 2022] PUT /identity/v3/projects/2faa26a1fcd54070a97ecce3a6a1d0fb/users/d5e8d59842a1403f95f3f31cc5a6b520/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:34.187259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53294b1-ea36-4745-9c56-37807d79d448 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:34.198685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53294b1-ea36-4745-9c56-37807d79d448 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:34.232456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:34.259548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46775587-9b42-46cf-a6c5-1e57060ef2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:34.260176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46775587-9b42-46cf-a6c5-1e57060ef2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:34.260454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46775587-9b42-46cf-a6c5-1e57060ef2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:34.260725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46775587-9b42-46cf-a6c5-1e57060ef2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:34.261412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46775587-9b42-46cf-a6c5-1e57060ef2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:34.263036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46775587-9b42-46cf-a6c5-1e57060ef2b5 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:34.273729 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1259/2515] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:48:34.277768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c10efb-91dc-431a-9363-f1f182519420 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:34.289575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c10efb-91dc-431a-9363-f1f182519420 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:34.294516 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc05dec-f5b4-479c-ae7d-d966322a40c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:34.295202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc05dec-f5b4-479c-ae7d-d966322a40c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:34.296005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc05dec-f5b4-479c-ae7d-d966322a40c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:34.296005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc05dec-f5b4-479c-ae7d-d966322a40c8 admin admin] PATH_INFO: `/v3/projects/2faa26a1fcd54070a97ecce3a6a1d0fb/users/d5e8d59842a1403f95f3f31cc5a6b520/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:34.297775 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc05dec-f5b4-479c-ae7d-d966322a40c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2faa26a1fcd54070a97ecce3a6a1d0fb, user_id=d5e8d59842a1403f95f3f31cc5a6b520, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:34.301037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc05dec-f5b4-479c-ae7d-d966322a40c8 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:34.311795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:34.313286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1257/2516] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:48:34 2022] PUT /identity/v3/projects/2faa26a1fcd54070a97ecce3a6a1d0fb/users/d5e8d59842a1403f95f3f31cc5a6b520/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:35.520367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520fd2-6a5b-43b9-a673-1410c7d4bd98 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:35.521293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520fd2-6a5b-43b9-a673-1410c7d4bd98 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:35.521293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520fd2-6a5b-43b9-a673-1410c7d4bd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:35.555845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-fc520fd2-6a5b-43b9-a673-1410c7d4bd98 None None] MFA Rules not processed for user `d5e8d59842a1403f95f3f31cc5a6b520`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:35.561471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-fc520fd2-6a5b-43b9-a673-1410c7d4bd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:35.633849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1260/2517] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:35.645763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc05dec-f5b4-479c-ae7d-d966322a40c8 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:35.647579 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-dfc05dec-f5b4-479c-ae7d-d966322a40c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:35.708122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc05dec-f5b4-479c-ae7d-d966322a40c8 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:35.711838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e294f8-6dc7-4c27-b525-10aef4e66573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:35.712458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e294f8-6dc7-4c27-b525-10aef4e66573 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:35.712738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e294f8-6dc7-4c27-b525-10aef4e66573 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:35.713011 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e294f8-6dc7-4c27-b525-10aef4e66573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:35.773795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e294f8-6dc7-4c27-b525-10aef4e66573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:35.775190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e294f8-6dc7-4c27-b525-10aef4e66573 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:35.785651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1258/2518] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:38.689564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc520fd2-6a5b-43b9-a673-1410c7d4bd98 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:38.699189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc520fd2-6a5b-43b9-a673-1410c7d4bd98 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:38.702794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02104db-488f-4ca7-a3af-a15070919750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:38.703405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02104db-488f-4ca7-a3af-a15070919750 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:38.703686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02104db-488f-4ca7-a3af-a15070919750 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:38.703959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02104db-488f-4ca7-a3af-a15070919750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:38.713959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02104db-488f-4ca7-a3af-a15070919750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:38.715412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02104db-488f-4ca7-a3af-a15070919750 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:38.725903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1261/2519] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:45.780855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=48 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:48.720692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=52 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:57.671346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e294f8-6dc7-4c27-b525-10aef4e66573 service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:57.678210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-67e294f8-6dc7-4c27-b525-10aef4e66573 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:57.768278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e294f8-6dc7-4c27-b525-10aef4e66573 service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:57.774228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366dfa5-15cf-4071-a99a-35624a50c3eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:57.774868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366dfa5-15cf-4071-a99a-35624a50c3eb service placement] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:57.775127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366dfa5-15cf-4071-a99a-35624a50c3eb service placement] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:57.775397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366dfa5-15cf-4071-a99a-35624a50c3eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:57.777170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1366dfa5-15cf-4071-a99a-35624a50c3eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:57.790293 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366dfa5-15cf-4071-a99a-35624a50c3eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:57.796371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366dfa5-15cf-4071-a99a-35624a50c3eb service placement] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:57.799814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02104db-488f-4ca7-a3af-a15070919750 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:57.804468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1259/2520] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:48:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:48:57.864126 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02104db-488f-4ca7-a3af-a15070919750 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:57.868639 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:57.869266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:57.869547 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:57.869816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] PATH_INFO: `/v3/users/e4c84ab662d74f249e29644d6bd28fcd` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:57.870991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c84ab662d74f249e29644d6bd28fcd)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:57.873141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:58.128059 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c84ab662d74f249e29644d6bd28fcd'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:58.169634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c84ab662d74f249e29644d6bd28fcd'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:58.182787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c84ab662d74f249e29644d6bd28fcd'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:58.183483 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c84ab662d74f249e29644d6bd28fcd'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:58.190446 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c84ab662d74f249e29644d6bd28fcd'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:58.197038 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1262/2521] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:48:57 2022] DELETE /identity/v3/users/e4c84ab662d74f249e29644d6bd28fcd => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:48:58.428276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366dfa5-15cf-4071-a99a-35624a50c3eb service placement] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:58.439326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366dfa5-15cf-4071-a99a-35624a50c3eb service placement] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:58.500256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:58.500893 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:58.501173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:58.501446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] PATH_INFO: `/v3/projects/9cbf429d461042238296518f0884c689` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:58.502624 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cbf429d461042238296518f0884c689)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:58.505320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:58.622451 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cbf429d461042238296518f0884c689', 'request_id': 'req-1c8bfda4-ba09-44d5-a21f-3afa07f48801'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:58.628712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cbf429d461042238296518f0884c689', 'request_id': 'req-1c8bfda4-ba09-44d5-a21f-3afa07f48801'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:58.689832 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cbf429d461042238296518f0884c689', 'request_id': 'req-1c8bfda4-ba09-44d5-a21f-3afa07f48801'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:58.698365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:58.702228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] The token cache is being invalidate because project 9cbf429d461042238296518f0884c689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:48:58.705328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:58.705328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:58.705328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1260/2522] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:48:58 2022] DELETE /identity/v3/projects/9cbf429d461042238296518f0884c689 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:58.709569 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:58.711047 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:58.724696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82b73f-833e-4c98-b8b2-3ce978c95d97 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:58.782477 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c90bd-c04a-4096-974d-b30f72009ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:58.783127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c90bd-c04a-4096-974d-b30f72009ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:58.783413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c90bd-c04a-4096-974d-b30f72009ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:58.783697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c90bd-c04a-4096-974d-b30f72009ad5 admin admin] PATH_INFO: `/v3/users/d5e8d59842a1403f95f3f31cc5a6b520` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:58.784881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c90bd-c04a-4096-974d-b30f72009ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5e8d59842a1403f95f3f31cc5a6b520)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:58.786891 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c90bd-c04a-4096-974d-b30f72009ad5 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:58.808550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:48:58.978832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-834c90bd-c04a-4096-974d-b30f72009ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5e8d59842a1403f95f3f31cc5a6b520'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:59.021807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-834c90bd-c04a-4096-974d-b30f72009ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5e8d59842a1403f95f3f31cc5a6b520'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:59.034429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-834c90bd-c04a-4096-974d-b30f72009ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5e8d59842a1403f95f3f31cc5a6b520'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:59.035119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-834c90bd-c04a-4096-974d-b30f72009ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5e8d59842a1403f95f3f31cc5a6b520'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:59.042630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-834c90bd-c04a-4096-974d-b30f72009ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5e8d59842a1403f95f3f31cc5a6b520'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:59.048646 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1263/2523] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:48:58 2022] DELETE /identity/v3/users/d5e8d59842a1403f95f3f31cc5a6b520 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:59.303608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:59.305142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:59.319873 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bfda4-ba09-44d5-a21f-3afa07f48801 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:59.385440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:59.386149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:59.386438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:59.386716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] PATH_INFO: `/v3/projects/2faa26a1fcd54070a97ecce3a6a1d0fb` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:59.388017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2faa26a1fcd54070a97ecce3a6a1d0fb)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:59.390031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:59.531831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2faa26a1fcd54070a97ecce3a6a1d0fb', 'request_id': 'req-442bf00b-cd6b-4a37-820a-f34823b9c897'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:59.539785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2faa26a1fcd54070a97ecce3a6a1d0fb', 'request_id': 'req-442bf00b-cd6b-4a37-820a-f34823b9c897'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:59.633067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2faa26a1fcd54070a97ecce3a6a1d0fb', 'request_id': 'req-442bf00b-cd6b-4a37-820a-f34823b9c897'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:59.645919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] The token cache is being invalidate because project 2faa26a1fcd54070a97ecce3a6a1d0fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:48:59.645919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:59.646666 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-442bf00b-cd6b-4a37-820a-f34823b9c897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:48:59.648265 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1261/2524] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:48:59 2022] DELETE /identity/v3/projects/2faa26a1fcd54070a97ecce3a6a1d0fb => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:48:59.653317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16bfde-343f-4277-9dbc-80c42aa45aae None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:59.653606 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16bfde-343f-4277-9dbc-80c42aa45aae None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:59.653881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16bfde-343f-4277-9dbc-80c42aa45aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:59.674569 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ac16bfde-343f-4277-9dbc-80c42aa45aae None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:59.675980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ac16bfde-343f-4277-9dbc-80c42aa45aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:59.737143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1264/2525] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:59.745278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69d986-3a00-49a9-a7bc-900b067b3d77 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:59.745789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69d986-3a00-49a9-a7bc-900b067b3d77 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:59.745907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69d986-3a00-49a9-a7bc-900b067b3d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:59.765781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0a69d986-3a00-49a9-a7bc-900b067b3d77 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:48:59.767045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0a69d986-3a00-49a9-a7bc-900b067b3d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:48:59.775771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1262/2526] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:48:59.780061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16bfde-343f-4277-9dbc-80c42aa45aae None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:59.790655 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16bfde-343f-4277-9dbc-80c42aa45aae None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:59.795385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671834e-604b-4988-b6d5-597dd3ec9a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:59.796010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671834e-604b-4988-b6d5-597dd3ec9a4a admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:59.796290 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671834e-604b-4988-b6d5-597dd3ec9a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:59.796560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671834e-604b-4988-b6d5-597dd3ec9a4a admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:59.797266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671834e-604b-4988-b6d5-597dd3ec9a4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:48:59.797570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671834e-604b-4988-b6d5-597dd3ec9a4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:59.799261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671834e-604b-4988-b6d5-597dd3ec9a4a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:59.814884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1265/2527] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:48:59.816543 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69d986-3a00-49a9-a7bc-900b067b3d77 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:59.826206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69d986-3a00-49a9-a7bc-900b067b3d77 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:59.830880 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b320259-0980-457c-9eda-0daa1c5f20aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:59.831496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320259-0980-457c-9eda-0daa1c5f20aa admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:59.831778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320259-0980-457c-9eda-0daa1c5f20aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:59.832587 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320259-0980-457c-9eda-0daa1c5f20aa admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:59.833505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b320259-0980-457c-9eda-0daa1c5f20aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:59.835446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b320259-0980-457c-9eda-0daa1c5f20aa admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:48:59.897839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1263/2528] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:48:59 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:48:59.902206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671834e-604b-4988-b6d5-597dd3ec9a4a admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:48:59.913438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671834e-604b-4988-b6d5-597dd3ec9a4a admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:48:59.970181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9184669-e66a-4755-94ec-d1ffc9a11115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:48:59.970826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9184669-e66a-4755-94ec-d1ffc9a11115 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:48:59.971093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9184669-e66a-4755-94ec-d1ffc9a11115 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:48:59.971372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9184669-e66a-4755-94ec-d1ffc9a11115 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:48:59.972045 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9184669-e66a-4755-94ec-d1ffc9a11115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:48:59.974613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9184669-e66a-4755-94ec-d1ffc9a11115 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:00.061864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1266/2529] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:48:59 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:49:00.066116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b320259-0980-457c-9eda-0daa1c5f20aa admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:00.077287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b320259-0980-457c-9eda-0daa1c5f20aa admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:00.082658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f38fc9-8d88-4868-84a6-7e5b4cc19512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:00.083305 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f38fc9-8d88-4868-84a6-7e5b4cc19512 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:00.083582 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f38fc9-8d88-4868-84a6-7e5b4cc19512 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:00.083851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f38fc9-8d88-4868-84a6-7e5b4cc19512 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:00.084545 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f38fc9-8d88-4868-84a6-7e5b4cc19512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:00.086174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f38fc9-8d88-4868-84a6-7e5b4cc19512 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:00.096595 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1264/2530] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:49:00.100822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9184669-e66a-4755-94ec-d1ffc9a11115 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:00.111208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9184669-e66a-4755-94ec-d1ffc9a11115 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:00.116501 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76bca6-9027-4750-884d-13ed9d77987b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:00.117135 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76bca6-9027-4750-884d-13ed9d77987b admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:00.117420 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76bca6-9027-4750-884d-13ed9d77987b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:00.117695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76bca6-9027-4750-884d-13ed9d77987b admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:00.118441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76bca6-9027-4750-884d-13ed9d77987b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:00.120052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76bca6-9027-4750-884d-13ed9d77987b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:00.132507 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1267/2531] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:49:00.136702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f38fc9-8d88-4868-84a6-7e5b4cc19512 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:00.147816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f38fc9-8d88-4868-84a6-7e5b4cc19512 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:00.152578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40328883-eae6-4797-957f-6bc8e7aa090b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:00.153251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40328883-eae6-4797-957f-6bc8e7aa090b admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:00.153532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40328883-eae6-4797-957f-6bc8e7aa090b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:00.153805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40328883-eae6-4797-957f-6bc8e7aa090b admin admin] PATH_INFO: `/v3/projects/acabec25d3b04636ba7d320c3c652642/users/7acb09474b724d9789f5d5c78e9f17ac/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:00.169741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40328883-eae6-4797-957f-6bc8e7aa090b admin admin] RBAC: Authorizing `identity:create_grant(project_id=acabec25d3b04636ba7d320c3c652642, user_id=7acb09474b724d9789f5d5c78e9f17ac, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:00.172483 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40328883-eae6-4797-957f-6bc8e7aa090b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:00.222152 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1265/2532] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:49:00 2022] PUT /identity/v3/projects/acabec25d3b04636ba7d320c3c652642/users/7acb09474b724d9789f5d5c78e9f17ac/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:49:00.233086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9af4f5-cae8-41f3-95eb-a9b311a377e8 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:00.233369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9af4f5-cae8-41f3-95eb-a9b311a377e8 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:00.233650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9af4f5-cae8-41f3-95eb-a9b311a377e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:00.242245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:00.264138 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1a9af4f5-cae8-41f3-95eb-a9b311a377e8 None None] MFA Rules not processed for user `7acb09474b724d9789f5d5c78e9f17ac`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:00.265353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1a9af4f5-cae8-41f3-95eb-a9b311a377e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:00.334214 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1268/2533] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:00.345171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40328883-eae6-4797-957f-6bc8e7aa090b admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:00.347094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-40328883-eae6-4797-957f-6bc8e7aa090b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:00.380712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:00.437423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40328883-eae6-4797-957f-6bc8e7aa090b admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:00.442109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d8024-3f2f-4fff-b88f-bde9641a3b60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:00.442815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d8024-3f2f-4fff-b88f-bde9641a3b60 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:00.443123 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d8024-3f2f-4fff-b88f-bde9641a3b60 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:00.443461 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d8024-3f2f-4fff-b88f-bde9641a3b60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:00.454326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d8024-3f2f-4fff-b88f-bde9641a3b60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:00.456043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d8024-3f2f-4fff-b88f-bde9641a3b60 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:00.467799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1266/2534] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:49:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:49:00.493923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9af4f5-cae8-41f3-95eb-a9b311a377e8 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:00.559904 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9af4f5-cae8-41f3-95eb-a9b311a377e8 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:00.564610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:00.565389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:00.565675 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:00.565946 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] PATH_INFO: `/v3/users/7acb09474b724d9789f5d5c78e9f17ac` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:00.567969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7acb09474b724d9789f5d5c78e9f17ac)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:00.569859 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:00.719566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7acb09474b724d9789f5d5c78e9f17ac'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:00.769147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7acb09474b724d9789f5d5c78e9f17ac'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:00.780934 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7acb09474b724d9789f5d5c78e9f17ac'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:00.788776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7acb09474b724d9789f5d5c78e9f17ac'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:00.788924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7acb09474b724d9789f5d5c78e9f17ac'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:00.794819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1269/2535] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:49:00 2022] DELETE /identity/v3/users/7acb09474b724d9789f5d5c78e9f17ac => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:49:00.799424 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec4ed7-31c9-45c9-8b44-2a7d5779d8bd None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:00.799936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec4ed7-31c9-45c9-8b44-2a7d5779d8bd None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:00.800049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec4ed7-31c9-45c9-8b44-2a7d5779d8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:00.819695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-67ec4ed7-31c9-45c9-8b44-2a7d5779d8bd None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:00.823259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-67ec4ed7-31c9-45c9-8b44-2a7d5779d8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:00.880626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1267/2536] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:00.890792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:00.892326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:00.910930 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:00.969161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e7bb1-267e-4cc1-bd3f-2288a75964e8 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:00.973637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e67e6-2cc8-4915-a137-b7414f9bce00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:00.974235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e67e6-2cc8-4915-a137-b7414f9bce00 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:00.974502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e67e6-2cc8-4915-a137-b7414f9bce00 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:00.974771 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e67e6-2cc8-4915-a137-b7414f9bce00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:00.984183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e67e6-2cc8-4915-a137-b7414f9bce00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:00.985591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e67e6-2cc8-4915-a137-b7414f9bce00 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:00.995645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1270/2537] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:49:01.393780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec4ed7-31c9-45c9-8b44-2a7d5779d8bd None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:01.395968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-67ec4ed7-31c9-45c9-8b44-2a7d5779d8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:01.409219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec4ed7-31c9-45c9-8b44-2a7d5779d8bd None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:01.414948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:01.419783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:01.419783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:01.419783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] PATH_INFO: `/v3/projects/acabec25d3b04636ba7d320c3c652642` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:01.419783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acabec25d3b04636ba7d320c3c652642)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:01.419783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:01.542489 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acabec25d3b04636ba7d320c3c652642', 'request_id': 'req-4fc4a960-091b-4d90-bba4-15a5fef60928'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:01.548876 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acabec25d3b04636ba7d320c3c652642', 'request_id': 'req-4fc4a960-091b-4d90-bba4-15a5fef60928'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:01.597866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acabec25d3b04636ba7d320c3c652642', 'request_id': 'req-4fc4a960-091b-4d90-bba4-15a5fef60928'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:01.609986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] The token cache is being invalidate because project acabec25d3b04636ba7d320c3c652642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:49:01.610299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:01.611058 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4fc4a960-091b-4d90-bba4-15a5fef60928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:01.613142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1268/2538] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:49:01 2022] DELETE /identity/v3/projects/acabec25d3b04636ba7d320c3c652642 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:49:01.618071 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428b7d0-d05f-4377-81f6-732c30e3e344 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:01.618381 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428b7d0-d05f-4377-81f6-732c30e3e344 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:01.618636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428b7d0-d05f-4377-81f6-732c30e3e344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:01.641684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-0428b7d0-d05f-4377-81f6-732c30e3e344 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:01.643194 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-0428b7d0-d05f-4377-81f6-732c30e3e344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:01.703052 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1271/2539] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:01.712330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecb824-af23-46a1-993f-57edaf343600 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:01.712619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecb824-af23-46a1-993f-57edaf343600 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:01.712905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecb824-af23-46a1-993f-57edaf343600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:01.732265 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-63ecb824-af23-46a1-993f-57edaf343600 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:01.733459 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-63ecb824-af23-46a1-993f-57edaf343600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:01.741718 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1269/2540] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:01.745892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428b7d0-d05f-4377-81f6-732c30e3e344 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:01.756186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428b7d0-d05f-4377-81f6-732c30e3e344 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:01.760796 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412d0ab-7644-400b-97cd-b47784349268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:01.761397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412d0ab-7644-400b-97cd-b47784349268 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:01.761850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412d0ab-7644-400b-97cd-b47784349268 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:01.762160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412d0ab-7644-400b-97cd-b47784349268 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:01.762844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412d0ab-7644-400b-97cd-b47784349268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:49:01.763148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412d0ab-7644-400b-97cd-b47784349268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:01.764723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412d0ab-7644-400b-97cd-b47784349268 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:01.776974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1272/2541] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:49:01.781217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecb824-af23-46a1-993f-57edaf343600 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:01.790717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecb824-af23-46a1-993f-57edaf343600 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:01.795401 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44cc0f-de94-4e22-831e-a0413e1e97ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:01.796004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44cc0f-de94-4e22-831e-a0413e1e97ab admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:01.796290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44cc0f-de94-4e22-831e-a0413e1e97ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:01.796569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44cc0f-de94-4e22-831e-a0413e1e97ab admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:01.797235 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44cc0f-de94-4e22-831e-a0413e1e97ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:01.799186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44cc0f-de94-4e22-831e-a0413e1e97ab admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:01.885390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1270/2542] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:49:01 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:49:01.889660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412d0ab-7644-400b-97cd-b47784349268 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:01.901043 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412d0ab-7644-400b-97cd-b47784349268 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:01.956181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14423e94-529c-4cd8-b7c3-6b74bdb3631a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:01.956791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14423e94-529c-4cd8-b7c3-6b74bdb3631a admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:01.957153 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14423e94-529c-4cd8-b7c3-6b74bdb3631a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:01.957429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14423e94-529c-4cd8-b7c3-6b74bdb3631a admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:01.958160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14423e94-529c-4cd8-b7c3-6b74bdb3631a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:01.960074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14423e94-529c-4cd8-b7c3-6b74bdb3631a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:02.020587 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1273/2543] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:49:01 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:49:02.024613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44cc0f-de94-4e22-831e-a0413e1e97ab admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:02.031409 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:02.035060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44cc0f-de94-4e22-831e-a0413e1e97ab admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:02.039713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bfe6e-4787-4e72-8d62-ce02ff808e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:02.040314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bfe6e-4787-4e72-8d62-ce02ff808e11 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:02.040596 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bfe6e-4787-4e72-8d62-ce02ff808e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:02.040869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bfe6e-4787-4e72-8d62-ce02ff808e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:02.041560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bfe6e-4787-4e72-8d62-ce02ff808e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:02.043179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bfe6e-4787-4e72-8d62-ce02ff808e11 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:02.054062 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1271/2544] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:49:02.057529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14423e94-529c-4cd8-b7c3-6b74bdb3631a admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:02.068297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14423e94-529c-4cd8-b7c3-6b74bdb3631a admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:02.073141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6789d4a-39e1-4471-ba1d-874f04003663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:02.073746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789d4a-39e1-4471-ba1d-874f04003663 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:02.074057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789d4a-39e1-4471-ba1d-874f04003663 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:02.074338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789d4a-39e1-4471-ba1d-874f04003663 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:02.075026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6789d4a-39e1-4471-ba1d-874f04003663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:02.076633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6789d4a-39e1-4471-ba1d-874f04003663 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:02.087343 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1274/2545] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:49:02.091389 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bfe6e-4787-4e72-8d62-ce02ff808e11 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:02.101533 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bfe6e-4787-4e72-8d62-ce02ff808e11 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:02.106578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5bc73-c5a4-470b-9ad4-77b6da68cd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:02.107227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5bc73-c5a4-470b-9ad4-77b6da68cd86 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:02.107508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5bc73-c5a4-470b-9ad4-77b6da68cd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:02.107777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5bc73-c5a4-470b-9ad4-77b6da68cd86 admin admin] PATH_INFO: `/v3/projects/ac0197e11045401e86ace8724d45e9d0/users/0b25f4619f424964b21a9a6efed56874/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:02.122949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5bc73-c5a4-470b-9ad4-77b6da68cd86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0197e11045401e86ace8724d45e9d0, user_id=0b25f4619f424964b21a9a6efed56874, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:02.125688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5bc73-c5a4-470b-9ad4-77b6da68cd86 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:02.186559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1272/2546] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:49:02 2022] PUT /identity/v3/projects/ac0197e11045401e86ace8724d45e9d0/users/0b25f4619f424964b21a9a6efed56874/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:49:02.197712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe962a2f-6178-4a63-a7ac-80ec7fa366b4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:02.197999 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe962a2f-6178-4a63-a7ac-80ec7fa366b4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:02.198302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe962a2f-6178-4a63-a7ac-80ec7fa366b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:02.205311 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:02.228247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-fe962a2f-6178-4a63-a7ac-80ec7fa366b4 None None] MFA Rules not processed for user `0b25f4619f424964b21a9a6efed56874`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:02.229518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-fe962a2f-6178-4a63-a7ac-80ec7fa366b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:02.295004 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1275/2547] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:02.306249 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5bc73-c5a4-470b-9ad4-77b6da68cd86 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:02.307916 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-56c5bc73-c5a4-470b-9ad4-77b6da68cd86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:02.364756 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5bc73-c5a4-470b-9ad4-77b6da68cd86 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:02.368988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8f2c0-de73-4838-9a2a-8f574695d8e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:02.369591 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f2c0-de73-4838-9a2a-8f574695d8e4 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:02.369870 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f2c0-de73-4838-9a2a-8f574695d8e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:02.370171 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f2c0-de73-4838-9a2a-8f574695d8e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:02.378597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8f2c0-de73-4838-9a2a-8f574695d8e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:02.380021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8f2c0-de73-4838-9a2a-8f574695d8e4 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:02.388608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1273/2548] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:49:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:49:12.292168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=31 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:12.384765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:13.374753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe962a2f-6178-4a63-a7ac-80ec7fa366b4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:13.440349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe962a2f-6178-4a63-a7ac-80ec7fa366b4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:13.449651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:13.449651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:13.449651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:13.449651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] PATH_INFO: `/v3/users/0b25f4619f424964b21a9a6efed56874` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:13.450105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b25f4619f424964b21a9a6efed56874)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:13.450105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:13.629352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b25f4619f424964b21a9a6efed56874'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:13.688897 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b25f4619f424964b21a9a6efed56874'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:13.701517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b25f4619f424964b21a9a6efed56874'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:13.702269 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b25f4619f424964b21a9a6efed56874'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:13.710684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b25f4619f424964b21a9a6efed56874'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:13.716568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1276/2549] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:49:13 2022] DELETE /identity/v3/users/0b25f4619f424964b21a9a6efed56874 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:49:13.721006 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc6e18-a7aa-40a9-8add-91c46337a8d7 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:13.721469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc6e18-a7aa-40a9-8add-91c46337a8d7 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:13.721559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc6e18-a7aa-40a9-8add-91c46337a8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:13.740811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-94fc6e18-a7aa-40a9-8add-91c46337a8d7 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:13.742022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-94fc6e18-a7aa-40a9-8add-91c46337a8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:13.800723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1274/2550] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:13.811610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:13.813277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:13.869440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a277b40-24a2-45fb-a493-c139275e58e8 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:13.873872 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4c1ac-35e6-4507-94cc-685f23f26c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:13.874526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c1ac-35e6-4507-94cc-685f23f26c89 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:13.874800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c1ac-35e6-4507-94cc-685f23f26c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:13.875068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c1ac-35e6-4507-94cc-685f23f26c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:13.884101 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4c1ac-35e6-4507-94cc-685f23f26c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:13.885508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4c1ac-35e6-4507-94cc-685f23f26c89 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:13.895573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1277/2551] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:49:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:49:14.297761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc6e18-a7aa-40a9-8add-91c46337a8d7 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:14.299317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-94fc6e18-a7aa-40a9-8add-91c46337a8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:14.312833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc6e18-a7aa-40a9-8add-91c46337a8d7 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:14.317761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:14.318589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:14.318880 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:14.319157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] PATH_INFO: `/v3/projects/ac0197e11045401e86ace8724d45e9d0` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:14.320393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac0197e11045401e86ace8724d45e9d0)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:14.323057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:14.426595 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac0197e11045401e86ace8724d45e9d0', 'request_id': 'req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:14.433113 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac0197e11045401e86ace8724d45e9d0', 'request_id': 'req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:14.506559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac0197e11045401e86ace8724d45e9d0', 'request_id': 'req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:14.530623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] The token cache is being invalidate because project ac0197e11045401e86ace8724d45e9d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:49:14.531245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:14.532350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-eca636a4-f3f3-4f90-b625-e04bd5fdc6ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:49:14.538124 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1275/2552] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:49:14 2022] DELETE /identity/v3/projects/ac0197e11045401e86ace8724d45e9d0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:49:14.539790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4dc11a-529c-477d-a32a-3ef9c301722f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:14.540590 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4dc11a-529c-477d-a32a-3ef9c301722f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:14.540880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4dc11a-529c-477d-a32a-3ef9c301722f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:14.566054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-6f4dc11a-529c-477d-a32a-3ef9c301722f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:14.567364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6f4dc11a-529c-477d-a32a-3ef9c301722f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:14.586448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:14.638935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1278/2553] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:49:14.648150 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a6a9f-0e3c-4ae5-9db1-3ea15ded87e0 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:14.648441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a6a9f-0e3c-4ae5-9db1-3ea15ded87e0 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:14.648718 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a6a9f-0e3c-4ae5-9db1-3ea15ded87e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:14.668261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d95a6a9f-0e3c-4ae5-9db1-3ea15ded87e0 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:14.669564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d95a6a9f-0e3c-4ae5-9db1-3ea15ded87e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:14.677753 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1276/2554] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:14.682186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4dc11a-529c-477d-a32a-3ef9c301722f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:14.692393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4dc11a-529c-477d-a32a-3ef9c301722f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:14.696890 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e7d1-c459-4a06-95c6-bc01fd588cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:14.697488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e7d1-c459-4a06-95c6-bc01fd588cdf admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:14.697772 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e7d1-c459-4a06-95c6-bc01fd588cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:14.698061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e7d1-c459-4a06-95c6-bc01fd588cdf admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:14.698743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7e7d1-c459-4a06-95c6-bc01fd588cdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:49:14.699093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7e7d1-c459-4a06-95c6-bc01fd588cdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:14.701311 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7e7d1-c459-4a06-95c6-bc01fd588cdf admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:14.712541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1279/2555] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:49:14.717444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a6a9f-0e3c-4ae5-9db1-3ea15ded87e0 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:14.727724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a6a9f-0e3c-4ae5-9db1-3ea15ded87e0 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:14.732587 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fb704-e784-402a-a549-43b2f33b4275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:14.733191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fb704-e784-402a-a549-43b2f33b4275 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:14.733473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fb704-e784-402a-a549-43b2f33b4275 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:14.733748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fb704-e784-402a-a549-43b2f33b4275 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:14.734484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fb704-e784-402a-a549-43b2f33b4275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:14.736397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fb704-e784-402a-a549-43b2f33b4275 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:14.804639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1277/2556] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:49:14 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:49:14.809884 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e7d1-c459-4a06-95c6-bc01fd588cdf admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:14.821146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e7d1-c459-4a06-95c6-bc01fd588cdf admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:14.877980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1000e-ee74-4f90-a792-107fd698d550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:14.878627 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1000e-ee74-4f90-a792-107fd698d550 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:14.878911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1000e-ee74-4f90-a792-107fd698d550 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:14.879185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1000e-ee74-4f90-a792-107fd698d550 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:14.879844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1000e-ee74-4f90-a792-107fd698d550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:14.881743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1000e-ee74-4f90-a792-107fd698d550 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:14.963314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1280/2557] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:49:14 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:49:14.967631 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fb704-e784-402a-a549-43b2f33b4275 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:14.979570 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fb704-e784-402a-a549-43b2f33b4275 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:14.986468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a65d8e-ce07-41e3-b6c2-7867c45dd462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:14.987283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a65d8e-ce07-41e3-b6c2-7867c45dd462 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:14.987658 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a65d8e-ce07-41e3-b6c2-7867c45dd462 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:14.988020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a65d8e-ce07-41e3-b6c2-7867c45dd462 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:14.988834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a65d8e-ce07-41e3-b6c2-7867c45dd462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:14.990471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a65d8e-ce07-41e3-b6c2-7867c45dd462 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:15.001560 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1278/2558] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:49:15.005571 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1000e-ee74-4f90-a792-107fd698d550 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:15.016115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1000e-ee74-4f90-a792-107fd698d550 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:15.020961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc4409-8a64-4326-99d0-7b9cc4822052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:15.021562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc4409-8a64-4326-99d0-7b9cc4822052 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:15.021842 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc4409-8a64-4326-99d0-7b9cc4822052 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:15.022152 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc4409-8a64-4326-99d0-7b9cc4822052 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:15.022862 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc4409-8a64-4326-99d0-7b9cc4822052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:15.024457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc4409-8a64-4326-99d0-7b9cc4822052 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:15.035888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1281/2559] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:49:15.040123 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a65d8e-ce07-41e3-b6c2-7867c45dd462 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:15.051209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a65d8e-ce07-41e3-b6c2-7867c45dd462 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:15.055821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a5b14-3250-4cad-bec4-491cea9aa6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:15.056471 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a5b14-3250-4cad-bec4-491cea9aa6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:15.056751 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a5b14-3250-4cad-bec4-491cea9aa6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:15.057024 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a5b14-3250-4cad-bec4-491cea9aa6ee admin admin] PATH_INFO: `/v3/projects/b1685a61dd0844269efb39407dcd1e67/users/11ce89a0a9254498977b9ef531aa3b2e/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:15.072494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a5b14-3250-4cad-bec4-491cea9aa6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1685a61dd0844269efb39407dcd1e67, user_id=11ce89a0a9254498977b9ef531aa3b2e, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:15.075541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a5b14-3250-4cad-bec4-491cea9aa6ee admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:15.152852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1279/2560] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:49:15 2022] PUT /identity/v3/projects/b1685a61dd0844269efb39407dcd1e67/users/11ce89a0a9254498977b9ef531aa3b2e/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:49:15.157974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c518f-81b2-49b1-b3d3-853305cbeec2 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:15.158299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c518f-81b2-49b1-b3d3-853305cbeec2 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:15.158576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c518f-81b2-49b1-b3d3-853305cbeec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:15.178229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-694c518f-81b2-49b1-b3d3-853305cbeec2 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:15.180134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-694c518f-81b2-49b1-b3d3-853305cbeec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:15.242641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1282/2561] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:15.252524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a5b14-3250-4cad-bec4-491cea9aa6ee admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:15.254314 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d30a5b14-3250-4cad-bec4-491cea9aa6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:15.276137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:15.312086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a5b14-3250-4cad-bec4-491cea9aa6ee admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:15.315810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c13e94-a286-4627-a008-d32fa7d84e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:15.316435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c13e94-a286-4627-a008-d32fa7d84e5d service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:15.316719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c13e94-a286-4627-a008-d32fa7d84e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:15.316995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c13e94-a286-4627-a008-d32fa7d84e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:15.326786 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c13e94-a286-4627-a008-d32fa7d84e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:15.328153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c13e94-a286-4627-a008-d32fa7d84e5d service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:15.338363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1280/2562] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:49:17.626496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670a0f6-d379-4a7d-b1af-99fcbe398e05 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:17.627085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670a0f6-d379-4a7d-b1af-99fcbe398e05 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:17.627085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670a0f6-d379-4a7d-b1af-99fcbe398e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:17.665277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-3670a0f6-d379-4a7d-b1af-99fcbe398e05 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:17.666636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3670a0f6-d379-4a7d-b1af-99fcbe398e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:17.681180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:17.734584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1283/2563] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:49:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:17.842768 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c13e94-a286-4627-a008-d32fa7d84e5d service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:17.844237 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b1c13e94-a286-4627-a008-d32fa7d84e5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:17.917366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c13e94-a286-4627-a008-d32fa7d84e5d service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:17.925070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e259db-e36f-444a-b7a2-8ec4963d6fea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:17.925855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e259db-e36f-444a-b7a2-8ec4963d6fea service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:17.926428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e259db-e36f-444a-b7a2-8ec4963d6fea service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:17.926711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e259db-e36f-444a-b7a2-8ec4963d6fea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:17.940900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e259db-e36f-444a-b7a2-8ec4963d6fea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:17.942757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e259db-e36f-444a-b7a2-8ec4963d6fea service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:17.952055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1281/2564] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:49:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:49:21.691604 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a629ee-bf6f-4852-a70d-9d57c3d5ee1e None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:21.692205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a629ee-bf6f-4852-a70d-9d57c3d5ee1e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:21.692205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a629ee-bf6f-4852-a70d-9d57c3d5ee1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:21.734573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-15a629ee-bf6f-4852-a70d-9d57c3d5ee1e None None] MFA Rules not processed for user `11ce89a0a9254498977b9ef531aa3b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:21.735987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-15a629ee-bf6f-4852-a70d-9d57c3d5ee1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:21.829239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1284/2565] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:21.833938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e259db-e36f-444a-b7a2-8ec4963d6fea service swift] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:21.848212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e259db-e36f-444a-b7a2-8ec4963d6fea service swift] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:21.854333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363cf2c-1cd6-46bb-abee-d4559dbdc25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:21.855814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363cf2c-1cd6-46bb-abee-d4559dbdc25a admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:21.856238 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363cf2c-1cd6-46bb-abee-d4559dbdc25a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:21.856638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363cf2c-1cd6-46bb-abee-d4559dbdc25a admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:21.857529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363cf2c-1cd6-46bb-abee-d4559dbdc25a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:21.859975 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363cf2c-1cd6-46bb-abee-d4559dbdc25a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:22.087162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1282/2566] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:49:21 2022] POST /identity/v3/projects => generated 385 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:49:22.091879 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a629ee-bf6f-4852-a70d-9d57c3d5ee1e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:22.104631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a629ee-bf6f-4852-a70d-9d57c3d5ee1e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:22.212627 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fc86e-d537-4b5f-814c-7baad1f989d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:22.217967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fc86e-d537-4b5f-814c-7baad1f989d4 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:22.217967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fc86e-d537-4b5f-814c-7baad1f989d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:22.217967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fc86e-d537-4b5f-814c-7baad1f989d4 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:22.217967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fc86e-d537-4b5f-814c-7baad1f989d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:22.217967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fc86e-d537-4b5f-814c-7baad1f989d4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:22.338974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1285/2567] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:49:22 2022] POST /identity/v3/users => generated 299 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:49:22.340794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363cf2c-1cd6-46bb-abee-d4559dbdc25a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:22.362031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363cf2c-1cd6-46bb-abee-d4559dbdc25a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:22.367638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7a4ef-d55e-4221-bbbd-6610aa621776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:22.368267 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7a4ef-d55e-4221-bbbd-6610aa621776 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:22.368548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7a4ef-d55e-4221-bbbd-6610aa621776 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:22.368822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7a4ef-d55e-4221-bbbd-6610aa621776 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:22.369521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7a4ef-d55e-4221-bbbd-6610aa621776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:22.371760 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7a4ef-d55e-4221-bbbd-6610aa621776 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:22.386091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1283/2568] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:49:22.389957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fc86e-d537-4b5f-814c-7baad1f989d4 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:22.400223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fc86e-d537-4b5f-814c-7baad1f989d4 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:22.406216 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a0e4b-88c9-4a2b-9d4b-5fc6742d570e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:22.406937 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0e4b-88c9-4a2b-9d4b-5fc6742d570e admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:22.407179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0e4b-88c9-4a2b-9d4b-5fc6742d570e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:22.407451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0e4b-88c9-4a2b-9d4b-5fc6742d570e admin admin] PATH_INFO: `/v3/projects/087cae9f6dda4ca5be96f9e185082e3a/users/b452be2babca422aa39e5763dc6b70c2/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:22.426702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a0e4b-88c9-4a2b-9d4b-5fc6742d570e admin admin] RBAC: Authorizing `identity:create_grant(project_id=087cae9f6dda4ca5be96f9e185082e3a, user_id=b452be2babca422aa39e5763dc6b70c2, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:22.429503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a0e4b-88c9-4a2b-9d4b-5fc6742d570e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:22.570840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1286/2569] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:49:22 2022] PUT /identity/v3/projects/087cae9f6dda4ca5be96f9e185082e3a/users/b452be2babca422aa39e5763dc6b70c2/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:49:22.576149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7a4ef-d55e-4221-bbbd-6610aa621776 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:22.595044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7a4ef-d55e-4221-bbbd-6610aa621776 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:22.690443 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f9e76-bb86-4b43-9482-e8a99a14b88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:22.691050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f9e76-bb86-4b43-9482-e8a99a14b88a admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:22.691333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f9e76-bb86-4b43-9482-e8a99a14b88a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:22.691605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f9e76-bb86-4b43-9482-e8a99a14b88a admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:22.692304 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f9e76-bb86-4b43-9482-e8a99a14b88a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:22.693932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f9e76-bb86-4b43-9482-e8a99a14b88a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:22.705089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1284/2570] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:49:22.710356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a0e4b-88c9-4a2b-9d4b-5fc6742d570e admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:22.722434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a0e4b-88c9-4a2b-9d4b-5fc6742d570e admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:22.727821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a065e-d15a-43da-bf18-e69f0b77920c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:22.728496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a065e-d15a-43da-bf18-e69f0b77920c admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:22.728779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a065e-d15a-43da-bf18-e69f0b77920c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:22.729062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a065e-d15a-43da-bf18-e69f0b77920c admin admin] PATH_INFO: `/v3/projects/087cae9f6dda4ca5be96f9e185082e3a/users/b452be2babca422aa39e5763dc6b70c2/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:22.732486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a065e-d15a-43da-bf18-e69f0b77920c admin admin] RBAC: Authorizing `identity:create_grant(project_id=087cae9f6dda4ca5be96f9e185082e3a, user_id=b452be2babca422aa39e5763dc6b70c2, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:22.736808 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a065e-d15a-43da-bf18-e69f0b77920c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:22.746765 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:22.752746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1287/2571] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:49:22 2022] PUT /identity/v3/projects/087cae9f6dda4ca5be96f9e185082e3a/users/b452be2babca422aa39e5763dc6b70c2/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:49:27.152986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f9e76-bb86-4b43-9482-e8a99a14b88a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:27.156407 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f68f9e76-bb86-4b43-9482-e8a99a14b88a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:27.180420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:27.267841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f9e76-bb86-4b43-9482-e8a99a14b88a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:27.371005 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2912511-60b9-4d95-b19c-3588a4cb2d85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:27.371988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2912511-60b9-4d95-b19c-3588a4cb2d85 service placement] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:27.372921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2912511-60b9-4d95-b19c-3588a4cb2d85 service placement] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:27.373668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2912511-60b9-4d95-b19c-3588a4cb2d85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:27.376205 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c2912511-60b9-4d95-b19c-3588a4cb2d85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:27.410937 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2912511-60b9-4d95-b19c-3588a4cb2d85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:27.412795 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2912511-60b9-4d95-b19c-3588a4cb2d85 service placement] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:27.454495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1285/2572] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:49:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:49:31.148516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd3430-524d-46a1-b1ec-f935ebc5891b None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:31.149039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd3430-524d-46a1-b1ec-f935ebc5891b None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:31.149158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd3430-524d-46a1-b1ec-f935ebc5891b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:31.340923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-e9cd3430-524d-46a1-b1ec-f935ebc5891b None None] MFA Rules not processed for user `b452be2babca422aa39e5763dc6b70c2`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:31.340923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-e9cd3430-524d-46a1-b1ec-f935ebc5891b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:31.340923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1288/2573] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:31.349858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2912511-60b9-4d95-b19c-3588a4cb2d85 service placement] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:31.351370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c2912511-60b9-4d95-b19c-3588a4cb2d85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:31.431473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2912511-60b9-4d95-b19c-3588a4cb2d85 service placement] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:31.436352 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4530a32-b8ed-491f-8dca-d7560717616d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a9a6bcfa1b469298774e09a8912b93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:31.436969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4530a32-b8ed-491f-8dca-d7560717616d service glance] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:31.437256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4530a32-b8ed-491f-8dca-d7560717616d service glance] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:31.437534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4530a32-b8ed-491f-8dca-d7560717616d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:31.502272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4530a32-b8ed-491f-8dca-d7560717616d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:31.503774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4530a32-b8ed-491f-8dca-d7560717616d service glance] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:31.513202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1286/2574] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:49:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:49:31.828975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab39a8-c4b2-4363-a97b-f036d91e8df5 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:31.829367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab39a8-c4b2-4363-a97b-f036d91e8df5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:31.829744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab39a8-c4b2-4363-a97b-f036d91e8df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:31.855010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bbab39a8-c4b2-4363-a97b-f036d91e8df5 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:31.857181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bbab39a8-c4b2-4363-a97b-f036d91e8df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:31.925947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1289/2575] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:49:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:31.945288 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4530a32-b8ed-491f-8dca-d7560717616d service glance] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:31.950707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:32.040886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4530a32-b8ed-491f-8dca-d7560717616d service glance] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:32.050625 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327662e-8c99-4244-9699-4404c75beb23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:32.051328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327662e-8c99-4244-9699-4404c75beb23 service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:32.051712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327662e-8c99-4244-9699-4404c75beb23 service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:32.052268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327662e-8c99-4244-9699-4404c75beb23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:32.068368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327662e-8c99-4244-9699-4404c75beb23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:32.070423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327662e-8c99-4244-9699-4404c75beb23 service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:32.080834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1287/2576] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:49:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:49:40.437642 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b3d1d-b8f5-4462-a0af-1cc887d18853 None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:40.438235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b3d1d-b8f5-4462-a0af-1cc887d18853 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:40.438235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b3d1d-b8f5-4462-a0af-1cc887d18853 None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:40.439572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1290/2577] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:49:40.447106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7191f-f1be-4fe9-b813-13917742f349 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:40.448009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7191f-f1be-4fe9-b813-13917742f349 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:40.448528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7191f-f1be-4fe9-b813-13917742f349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:40.507518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-3ef7191f-f1be-4fe9-b813-13917742f349 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:40.508827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3ef7191f-f1be-4fe9-b813-13917742f349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:40.617687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1288/2578] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:49:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:49:40.630561 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b3d1d-b8f5-4462-a0af-1cc887d18853 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:40.632530 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c44b3d1d-b8f5-4462-a0af-1cc887d18853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:40.647795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b3d1d-b8f5-4462-a0af-1cc887d18853 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:40.653615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4faf5-b784-4cb3-bf51-81a36a2944be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a9a6bcfa1b469298774e09a8912b93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:40.654443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4faf5-b784-4cb3-bf51-81a36a2944be service glance] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:40.654837 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4faf5-b784-4cb3-bf51-81a36a2944be service glance] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:40.655208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4faf5-b784-4cb3-bf51-81a36a2944be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:40.666673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4faf5-b784-4cb3-bf51-81a36a2944be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:40.668664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4faf5-b784-4cb3-bf51-81a36a2944be service glance] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:40.680928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1291/2579] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:49:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:49:40.893539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c556c-6df0-47ec-bf21-509204953921 None None] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:40.894050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c556c-6df0-47ec-bf21-509204953921 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:40.894460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c556c-6df0-47ec-bf21-509204953921 None None] PATH_INFO: `/` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:40.896153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1289/2580] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:49:40.901967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4a183-898e-413a-a31c-e44627374165 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:40.902315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4a183-898e-413a-a31c-e44627374165 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:40.902626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4a183-898e-413a-a31c-e44627374165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:40.926139 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-6fc4a183-898e-413a-a31c-e44627374165 None None] MFA Rules not processed for user `c98761f81496456fb3b5a5cffd4ce763`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:40.927368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6fc4a183-898e-413a-a31c-e44627374165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:40.988677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1292/2581] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:49:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:41.002487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c556c-6df0-47ec-bf21-509204953921 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:41.016245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c556c-6df0-47ec-bf21-509204953921 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:41.019848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2858ff-a546-4162-8320-3dd6c5c3fcda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:41.020461 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2858ff-a546-4162-8320-3dd6c5c3fcda service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:41.020746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2858ff-a546-4162-8320-3dd6c5c3fcda service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:41.021021 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2858ff-a546-4162-8320-3dd6c5c3fcda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:41.030966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2858ff-a546-4162-8320-3dd6c5c3fcda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:41.032419 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2858ff-a546-4162-8320-3dd6c5c3fcda service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:41.041236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1290/2582] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:49:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:49:41.139673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14acf-5ab3-4b9c-b054-902db26fe9ee None None] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:41.139959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14acf-5ab3-4b9c-b054-902db26fe9ee None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:41.140232 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14acf-5ab3-4b9c-b054-902db26fe9ee None None] PATH_INFO: `/` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:41.141578 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1293/2583] 10.222.0.73 () {54 vars in 921 bytes} [Thu Aug 4 21:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 21:49:41.147456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb64a3-02db-4a9c-ae8a-b3d1e18a8470 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:41.148016 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb64a3-02db-4a9c-ae8a-b3d1e18a8470 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:41.148448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb64a3-02db-4a9c-ae8a-b3d1e18a8470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:41.170308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c3cb64a3-02db-4a9c-ae8a-b3d1e18a8470 None None] MFA Rules not processed for user `e7b8267780364a2181abf5817b54ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:41.171690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c3cb64a3-02db-4a9c-ae8a-b3d1e18a8470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:41.182465 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1291/2584] 10.222.0.73 () {58 vars in 1018 bytes} [Thu Aug 4 21:49:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:41.202982 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec14acf-5ab3-4b9c-b054-902db26fe9ee None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:41.217550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec14acf-5ab3-4b9c-b054-902db26fe9ee None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:41.221212 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cf081-315a-4a85-9da9-f02865808cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:41.221830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cf081-315a-4a85-9da9-f02865808cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:41.222155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cf081-315a-4a85-9da9-f02865808cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:41.222437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cf081-315a-4a85-9da9-f02865808cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:41.236076 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62cf081-315a-4a85-9da9-f02865808cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:41.237542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62cf081-315a-4a85-9da9-f02865808cd0 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:41.253956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1294/2585] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:49:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:49:43.921995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa40e-5ba3-4f8f-9f14-502b6d9b2ea1 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:43.922627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa40e-5ba3-4f8f-9f14-502b6d9b2ea1 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:43.922627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa40e-5ba3-4f8f-9f14-502b6d9b2ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:43.941909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-36caa40e-5ba3-4f8f-9f14-502b6d9b2ea1 None None] MFA Rules not processed for user `11ce89a0a9254498977b9ef531aa3b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:43.943144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-36caa40e-5ba3-4f8f-9f14-502b6d9b2ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:43.950816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1292/2586] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:49:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:45.330328 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43426f-607f-473d-ba72-e0aa61e40729 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:45.330807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43426f-607f-473d-ba72-e0aa61e40729 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:45.331145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43426f-607f-473d-ba72-e0aa61e40729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:45.354326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ac43426f-607f-473d-ba72-e0aa61e40729 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:45.355596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ac43426f-607f-473d-ba72-e0aa61e40729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:45.363920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1295/2587] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:49:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:45.376470 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caa40e-5ba3-4f8f-9f14-502b6d9b2ea1 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:49:45.388879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caa40e-5ba3-4f8f-9f14-502b6d9b2ea1 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:49:45.394492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb129c-73d9-4e00-ac0c-0e8369e3eb35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:49:45.395116 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb129c-73d9-4e00-ac0c-0e8369e3eb35 service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:45.395426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb129c-73d9-4e00-ac0c-0e8369e3eb35 service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:45.395696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb129c-73d9-4e00-ac0c-0e8369e3eb35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:45.404834 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb129c-73d9-4e00-ac0c-0e8369e3eb35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:49:45.406437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb129c-73d9-4e00-ac0c-0e8369e3eb35 service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:49:45.415173 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1293/2588] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:49:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:49:55.348726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:55.411816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:49:59.951181 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167cc6f-f11c-4730-8927-0fac423cc44f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:49:59.951653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167cc6f-f11c-4730-8927-0fac423cc44f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:49:59.951786 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167cc6f-f11c-4730-8927-0fac423cc44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:49:59.972807 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-6167cc6f-f11c-4730-8927-0fac423cc44f None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:49:59.974031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6167cc6f-f11c-4730-8927-0fac423cc44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:49:59.981268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1296/2589] 10.222.0.73 () {58 vars in 1009 bytes} [Thu Aug 4 21:49:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:49:59.994231 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb129c-73d9-4e00-ac0c-0e8369e3eb35 service swift] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:00.005824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb129c-73d9-4e00-ac0c-0e8369e3eb35 service swift] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:00.011008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b381572-d2c7-48e7-974c-fca67ebf7a3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:00.011652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b381572-d2c7-48e7-974c-fca67ebf7a3d service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:00.011935 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b381572-d2c7-48e7-974c-fca67ebf7a3d service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:00.012213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b381572-d2c7-48e7-974c-fca67ebf7a3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:00.022263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b381572-d2c7-48e7-974c-fca67ebf7a3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:00.023837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b381572-d2c7-48e7-974c-fca67ebf7a3d service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:00.032778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1294/2590] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:49:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:50:09.967023 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:10.028618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=6 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:12.509048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e2a7a-c213-4674-88f8-d299b6782012 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:12.509497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e2a7a-c213-4674-88f8-d299b6782012 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:12.509701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e2a7a-c213-4674-88f8-d299b6782012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:12.530086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-8f4e2a7a-c213-4674-88f8-d299b6782012 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:12.532254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8f4e2a7a-c213-4674-88f8-d299b6782012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:12.540005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1297/2591] 10.222.0.73 () {58 vars in 967 bytes} [Thu Aug 4 21:50:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:12.553216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b381572-d2c7-48e7-974c-fca67ebf7a3d service swift] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:12.565100 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b381572-d2c7-48e7-974c-fca67ebf7a3d service swift] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:12.570330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f9bf6-bd44-44e7-9b89-76341c6f5119 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:12.570988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f9bf6-bd44-44e7-9b89-76341c6f5119 service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:12.571271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f9bf6-bd44-44e7-9b89-76341c6f5119 service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:12.571780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f9bf6-bd44-44e7-9b89-76341c6f5119 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:12.581262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f9bf6-bd44-44e7-9b89-76341c6f5119 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:12.582990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f9bf6-bd44-44e7-9b89-76341c6f5119 service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:12.593548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1295/2592] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:50:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:50:17.278611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e2a7a-c213-4674-88f8-d299b6782012 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:17.280221 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8f4e2a7a-c213-4674-88f8-d299b6782012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:17.292288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e2a7a-c213-4674-88f8-d299b6782012 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:17.295893 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc6acd-2f81-4546-96b4-72d472eeefcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:17.296516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc6acd-2f81-4546-96b4-72d472eeefcc service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:17.296795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc6acd-2f81-4546-96b4-72d472eeefcc service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:17.298927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc6acd-2f81-4546-96b4-72d472eeefcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:17.304259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d4fc6acd-2f81-4546-96b4-72d472eeefcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:17.318155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc6acd-2f81-4546-96b4-72d472eeefcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:17.320805 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc6acd-2f81-4546-96b4-72d472eeefcc service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:17.330677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1298/2593] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:50:18.138886 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f9bf6-bd44-44e7-9b89-76341c6f5119 service swift] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:18.151077 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f9bf6-bd44-44e7-9b89-76341c6f5119 service swift] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:18.161742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5cc3f-a009-4302-b345-e062bf267f3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:18.161742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5cc3f-a009-4302-b345-e062bf267f3f service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:18.161742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5cc3f-a009-4302-b345-e062bf267f3f service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:18.161742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5cc3f-a009-4302-b345-e062bf267f3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:18.162528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a9c5cc3f-a009-4302-b345-e062bf267f3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:18.183925 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5cc3f-a009-4302-b345-e062bf267f3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:18.183925 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5cc3f-a009-4302-b345-e062bf267f3f service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:18.193512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1296/2594] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:50:24.586403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc6acd-2f81-4546-96b4-72d472eeefcc service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:24.595020 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc6acd-2f81-4546-96b4-72d472eeefcc service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:24.598653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d75dc6-1759-4046-b848-9148696c71d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:24.599255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d75dc6-1759-4046-b848-9148696c71d6 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:24.599536 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d75dc6-1759-4046-b848-9148696c71d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:24.599811 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d75dc6-1759-4046-b848-9148696c71d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:24.601575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-36d75dc6-1759-4046-b848-9148696c71d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:24.664395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d75dc6-1759-4046-b848-9148696c71d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:24.665816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d75dc6-1759-4046-b848-9148696c71d6 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:24.675548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1299/2595] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:50:28.188090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=12 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:28.558271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5cc3f-a009-4302-b345-e062bf267f3f service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:28.560056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a9c5cc3f-a009-4302-b345-e062bf267f3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:28.575012 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5cc3f-a009-4302-b345-e062bf267f3f service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:28.579657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b423f-88f5-4952-a1a7-c62f917c7b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:28.580344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b423f-88f5-4952-a1a7-c62f917c7b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:28.580624 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b423f-88f5-4952-a1a7-c62f917c7b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:28.580900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b423f-88f5-4952-a1a7-c62f917c7b34 admin admin] PATH_INFO: `/v3/users/a23fcabad2354e7ab51a22b412bf11e6` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:28.594703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b423f-88f5-4952-a1a7-c62f917c7b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a23fcabad2354e7ab51a22b412bf11e6)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:28.596624 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b423f-88f5-4952-a1a7-c62f917c7b34 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:28.748285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-9b0b423f-88f5-4952-a1a7-c62f917c7b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a23fcabad2354e7ab51a22b412bf11e6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:28.819144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-9b0b423f-88f5-4952-a1a7-c62f917c7b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a23fcabad2354e7ab51a22b412bf11e6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:28.819774 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-9b0b423f-88f5-4952-a1a7-c62f917c7b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a23fcabad2354e7ab51a22b412bf11e6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:28.832207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-9b0b423f-88f5-4952-a1a7-c62f917c7b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a23fcabad2354e7ab51a22b412bf11e6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:28.839973 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-9b0b423f-88f5-4952-a1a7-c62f917c7b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a23fcabad2354e7ab51a22b412bf11e6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:28.845932 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1297/2596] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:50:28 2022] DELETE /identity/v3/users/a23fcabad2354e7ab51a22b412bf11e6 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:29.083943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d75dc6-1759-4046-b848-9148696c71d6 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:29.096067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d75dc6-1759-4046-b848-9148696c71d6 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:29.155672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:29.156311 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:29.156595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:29.156868 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] PATH_INFO: `/v3/projects/09370df0a2354bd2a412c6770d3ff165` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:29.167601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09370df0a2354bd2a412c6770d3ff165)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:29.169488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:29.264881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09370df0a2354bd2a412c6770d3ff165', 'request_id': 'req-371cc369-3f93-4e18-b0f3-9004fc77b656'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:29.327784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09370df0a2354bd2a412c6770d3ff165', 'request_id': 'req-371cc369-3f93-4e18-b0f3-9004fc77b656'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:29.333590 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09370df0a2354bd2a412c6770d3ff165', 'request_id': 'req-371cc369-3f93-4e18-b0f3-9004fc77b656'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:29.336256 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:29.346179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] The token cache is being invalidate because project 09370df0a2354bd2a412c6770d3ff165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:50:29.346459 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:29.347131 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-371cc369-3f93-4e18-b0f3-9004fc77b656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:29.348637 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1300/2597] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:50:29 2022] DELETE /identity/v3/projects/09370df0a2354bd2a412c6770d3ff165 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:29.354469 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e464718-eefc-49ca-8655-ed632ec9a41c None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:29.354947 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e464718-eefc-49ca-8655-ed632ec9a41c None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:29.355115 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e464718-eefc-49ca-8655-ed632ec9a41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:29.376825 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-2e464718-eefc-49ca-8655-ed632ec9a41c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:29.378119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2e464718-eefc-49ca-8655-ed632ec9a41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:29.439222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1298/2598] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:29.448771 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed28c08-ca33-43e6-b2a2-b42757a546b5 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:29.449094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed28c08-ca33-43e6-b2a2-b42757a546b5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:29.449391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed28c08-ca33-43e6-b2a2-b42757a546b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:29.469129 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-0ed28c08-ca33-43e6-b2a2-b42757a546b5 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:29.470406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-0ed28c08-ca33-43e6-b2a2-b42757a546b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:29.478344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1301/2599] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:29.482592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e464718-eefc-49ca-8655-ed632ec9a41c None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:29.492999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e464718-eefc-49ca-8655-ed632ec9a41c None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:29.498648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e53080-9093-4e07-b1c5-1d451fb8522a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:29.499262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e53080-9093-4e07-b1c5-1d451fb8522a admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:29.499547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e53080-9093-4e07-b1c5-1d451fb8522a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:29.499824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e53080-9093-4e07-b1c5-1d451fb8522a admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:29.500523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e53080-9093-4e07-b1c5-1d451fb8522a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:50:29.500832 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e53080-9093-4e07-b1c5-1d451fb8522a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:29.502488 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e53080-9093-4e07-b1c5-1d451fb8522a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:29.515557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1299/2600] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:50:29.523035 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed28c08-ca33-43e6-b2a2-b42757a546b5 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:29.533272 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed28c08-ca33-43e6-b2a2-b42757a546b5 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:29.542431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0804fc-a680-41d4-bd52-96514d07d961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:29.543060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0804fc-a680-41d4-bd52-96514d07d961 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:29.543338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0804fc-a680-41d4-bd52-96514d07d961 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:29.543632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0804fc-a680-41d4-bd52-96514d07d961 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:29.544340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0804fc-a680-41d4-bd52-96514d07d961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:29.546318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0804fc-a680-41d4-bd52-96514d07d961 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:29.661046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1302/2601] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:50:29 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:50:29.665777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e53080-9093-4e07-b1c5-1d451fb8522a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:29.676013 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e53080-9093-4e07-b1c5-1d451fb8522a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:29.720253 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:29.732430 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fc7fc-66f6-403d-9db6-7dcb30d3c807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:29.733038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fc7fc-66f6-403d-9db6-7dcb30d3c807 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:29.733322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fc7fc-66f6-403d-9db6-7dcb30d3c807 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:29.733598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fc7fc-66f6-403d-9db6-7dcb30d3c807 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:29.734385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fc7fc-66f6-403d-9db6-7dcb30d3c807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:29.736328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fc7fc-66f6-403d-9db6-7dcb30d3c807 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:29.813693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1300/2602] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:50:29 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:50:29.818028 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0804fc-a680-41d4-bd52-96514d07d961 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:29.828575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0804fc-a680-41d4-bd52-96514d07d961 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:29.833489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a8819-3c56-4f41-8555-5f7839d81d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:29.834219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a8819-3c56-4f41-8555-5f7839d81d19 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:29.834695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a8819-3c56-4f41-8555-5f7839d81d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:29.835159 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a8819-3c56-4f41-8555-5f7839d81d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:29.835858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a8819-3c56-4f41-8555-5f7839d81d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:29.837423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a8819-3c56-4f41-8555-5f7839d81d19 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:29.847245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1303/2603] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:50:29.851788 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fc7fc-66f6-403d-9db6-7dcb30d3c807 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:29.861986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fc7fc-66f6-403d-9db6-7dcb30d3c807 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:29.867330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f6433-2756-49a7-8d18-7ba18b3c56c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:29.867935 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f6433-2756-49a7-8d18-7ba18b3c56c7 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:29.868215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f6433-2756-49a7-8d18-7ba18b3c56c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:29.868486 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f6433-2756-49a7-8d18-7ba18b3c56c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:29.869164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f6433-2756-49a7-8d18-7ba18b3c56c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:29.870781 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f6433-2756-49a7-8d18-7ba18b3c56c7 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:29.882667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1301/2604] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:50:29.886925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a8819-3c56-4f41-8555-5f7839d81d19 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:29.897374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a8819-3c56-4f41-8555-5f7839d81d19 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:29.901938 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785e6dd-6761-4982-b788-81cd673ff45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:29.902662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785e6dd-6761-4982-b788-81cd673ff45b admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:29.902941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785e6dd-6761-4982-b788-81cd673ff45b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:29.903214 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785e6dd-6761-4982-b788-81cd673ff45b admin admin] PATH_INFO: `/v3/projects/55666f6e2a9240389068279a71d32a1c/users/0e5927bdd755425ca37db47fc1cbad41/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:29.918092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785e6dd-6761-4982-b788-81cd673ff45b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55666f6e2a9240389068279a71d32a1c, user_id=0e5927bdd755425ca37db47fc1cbad41, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:29.920700 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785e6dd-6761-4982-b788-81cd673ff45b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:29.979934 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1304/2605] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:50:29 2022] PUT /identity/v3/projects/55666f6e2a9240389068279a71d32a1c/users/0e5927bdd755425ca37db47fc1cbad41/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:29.990859 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb175b4-8acc-4a01-a853-0f87165fe1d1 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:29.991151 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb175b4-8acc-4a01-a853-0f87165fe1d1 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:29.991475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb175b4-8acc-4a01-a853-0f87165fe1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:30.019399 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-3fb175b4-8acc-4a01-a853-0f87165fe1d1 None None] MFA Rules not processed for user `0e5927bdd755425ca37db47fc1cbad41`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:30.020610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3fb175b4-8acc-4a01-a853-0f87165fe1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:30.086894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1302/2606] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:50:30.098300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785e6dd-6761-4982-b788-81cd673ff45b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:30.100947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3785e6dd-6761-4982-b788-81cd673ff45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:30.173617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785e6dd-6761-4982-b788-81cd673ff45b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:30.179814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75a444-d93d-4ab5-9c5a-183f2b55cc54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:30.180426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75a444-d93d-4ab5-9c5a-183f2b55cc54 service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:30.180705 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75a444-d93d-4ab5-9c5a-183f2b55cc54 service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:30.180976 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75a444-d93d-4ab5-9c5a-183f2b55cc54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:30.188723 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75a444-d93d-4ab5-9c5a-183f2b55cc54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:30.190318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75a444-d93d-4ab5-9c5a-183f2b55cc54 service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:30.199576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1305/2607] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:50:33.295918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb175b4-8acc-4a01-a853-0f87165fe1d1 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:33.359431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb175b4-8acc-4a01-a853-0f87165fe1d1 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:33.364165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:33.364866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:33.365148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:33.365426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] PATH_INFO: `/v3/users/0e5927bdd755425ca37db47fc1cbad41` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:33.367084 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e5927bdd755425ca37db47fc1cbad41)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:33.369427 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:33.514230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e5927bdd755425ca37db47fc1cbad41'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:33.642333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e5927bdd755425ca37db47fc1cbad41'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:33.643107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e5927bdd755425ca37db47fc1cbad41'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:33.648860 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:33.657217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e5927bdd755425ca37db47fc1cbad41'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:33.664167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e5927bdd755425ca37db47fc1cbad41'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:33.670147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1303/2608] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:50:33 2022] DELETE /identity/v3/users/0e5927bdd755425ca37db47fc1cbad41 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:33.675266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fd4ee-86b3-4949-af5c-11a58666fbfe None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:33.675750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fd4ee-86b3-4949-af5c-11a58666fbfe None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:33.675861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fd4ee-86b3-4949-af5c-11a58666fbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:33.697110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-955fd4ee-86b3-4949-af5c-11a58666fbfe None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:33.698463 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-955fd4ee-86b3-4949-af5c-11a58666fbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:33.763175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1306/2609] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:50:33.773821 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:33.775255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:33.833071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497abf7c-78d2-43ec-9e0a-18f8f9dd6113 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:33.837564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b523f0-9890-4f6c-ba31-d62d8e98011b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:33.838235 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b523f0-9890-4f6c-ba31-d62d8e98011b service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:33.838515 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b523f0-9890-4f6c-ba31-d62d8e98011b service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:33.838788 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b523f0-9890-4f6c-ba31-d62d8e98011b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:33.848148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b523f0-9890-4f6c-ba31-d62d8e98011b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:33.849571 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b523f0-9890-4f6c-ba31-d62d8e98011b service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:33.860415 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1304/2610] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:50:34.273462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fd4ee-86b3-4949-af5c-11a58666fbfe None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:34.274981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-955fd4ee-86b3-4949-af5c-11a58666fbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:34.287661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fd4ee-86b3-4949-af5c-11a58666fbfe None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:34.293414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:34.294091 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:34.294376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:34.294649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] PATH_INFO: `/v3/projects/55666f6e2a9240389068279a71d32a1c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:34.296024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55666f6e2a9240389068279a71d32a1c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:34.298795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:34.301434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:34.413407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55666f6e2a9240389068279a71d32a1c', 'request_id': 'req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:34.481617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55666f6e2a9240389068279a71d32a1c', 'request_id': 'req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:34.487710 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55666f6e2a9240389068279a71d32a1c', 'request_id': 'req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:34.501462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] The token cache is being invalidate because project 55666f6e2a9240389068279a71d32a1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:50:34.501789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:34.502462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-70c04578-bcdd-4c9e-8208-fa9c2f3dfe47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:34.504160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1307/2611] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:50:34 2022] DELETE /identity/v3/projects/55666f6e2a9240389068279a71d32a1c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:34.509723 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf3c4-35a7-44ee-830c-0e5ab9d791fc None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:34.510327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf3c4-35a7-44ee-830c-0e5ab9d791fc None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:34.510327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf3c4-35a7-44ee-830c-0e5ab9d791fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:34.530669 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1edbf3c4-35a7-44ee-830c-0e5ab9d791fc None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:34.531885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1edbf3c4-35a7-44ee-830c-0e5ab9d791fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:34.591392 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1305/2612] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:34.599394 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f515e73-0a06-4f81-b2ca-606b2e1865b5 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:34.599689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f515e73-0a06-4f81-b2ca-606b2e1865b5 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:34.599969 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f515e73-0a06-4f81-b2ca-606b2e1865b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:34.620067 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-4f515e73-0a06-4f81-b2ca-606b2e1865b5 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:34.621279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4f515e73-0a06-4f81-b2ca-606b2e1865b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:34.629456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1308/2613] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:34.634336 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbf3c4-35a7-44ee-830c-0e5ab9d791fc None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:34.644215 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbf3c4-35a7-44ee-830c-0e5ab9d791fc None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:34.649185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c194e9f-e815-4b19-87b4-7ba06e1f4231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:34.649798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c194e9f-e815-4b19-87b4-7ba06e1f4231 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:34.650104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c194e9f-e815-4b19-87b4-7ba06e1f4231 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:34.650381 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c194e9f-e815-4b19-87b4-7ba06e1f4231 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:34.651074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c194e9f-e815-4b19-87b4-7ba06e1f4231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:50:34.651377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c194e9f-e815-4b19-87b4-7ba06e1f4231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:34.652966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c194e9f-e815-4b19-87b4-7ba06e1f4231 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:34.665244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1306/2614] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:50:34.669987 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f515e73-0a06-4f81-b2ca-606b2e1865b5 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:34.679279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f515e73-0a06-4f81-b2ca-606b2e1865b5 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:34.684169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61dfb7-74e7-4804-8325-b4b1726902e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:34.684777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61dfb7-74e7-4804-8325-b4b1726902e3 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:34.685055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61dfb7-74e7-4804-8325-b4b1726902e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:34.685326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61dfb7-74e7-4804-8325-b4b1726902e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:34.685996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61dfb7-74e7-4804-8325-b4b1726902e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:34.687878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61dfb7-74e7-4804-8325-b4b1726902e3 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:34.764699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1309/2615] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:50:34 2022] POST /identity/v3/projects => generated 405 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:50:34.773395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c194e9f-e815-4b19-87b4-7ba06e1f4231 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:34.785167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c194e9f-e815-4b19-87b4-7ba06e1f4231 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:34.845217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c22277-bba5-4f52-9a65-38ebf9701e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:34.845899 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c22277-bba5-4f52-9a65-38ebf9701e1e admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:34.846242 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c22277-bba5-4f52-9a65-38ebf9701e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:34.846848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c22277-bba5-4f52-9a65-38ebf9701e1e admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:34.847536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c22277-bba5-4f52-9a65-38ebf9701e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:34.849472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c22277-bba5-4f52-9a65-38ebf9701e1e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:34.861333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:34.981396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1307/2616] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:50:34 2022] POST /identity/v3/users => generated 303 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:50:34.990404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61dfb7-74e7-4804-8325-b4b1726902e3 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:34.994797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c22277-bba5-4f52-9a65-38ebf9701e1e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:34.996681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-87c22277-bba5-4f52-9a65-38ebf9701e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:35.001206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61dfb7-74e7-4804-8325-b4b1726902e3 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:35.005986 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f27d0-4d86-4eee-86e7-19b65a41cb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:35.007369 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f27d0-4d86-4eee-86e7-19b65a41cb92 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.007651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f27d0-4d86-4eee-86e7-19b65a41cb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.007924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f27d0-4d86-4eee-86e7-19b65a41cb92 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.008623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f27d0-4d86-4eee-86e7-19b65a41cb92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:35.010276 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f27d0-4d86-4eee-86e7-19b65a41cb92 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:35.020419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1310/2617] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:50:35.029861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f27d0-4d86-4eee-86e7-19b65a41cb92 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:35.039642 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f27d0-4d86-4eee-86e7-19b65a41cb92 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:35.044273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec6264-1f5c-4bce-bcea-4bb99104864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:35.044870 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec6264-1f5c-4bce-bcea-4bb99104864f admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.045148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec6264-1f5c-4bce-bcea-4bb99104864f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.045419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec6264-1f5c-4bce-bcea-4bb99104864f admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.046197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec6264-1f5c-4bce-bcea-4bb99104864f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:35.047799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec6264-1f5c-4bce-bcea-4bb99104864f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:35.054938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c22277-bba5-4f52-9a65-38ebf9701e1e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:35.063869 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1311/2618] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:50:35.063991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424e752-ec34-4e59-bd77-1e8be1d8ed48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:35.063991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424e752-ec34-4e59-bd77-1e8be1d8ed48 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.063991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424e752-ec34-4e59-bd77-1e8be1d8ed48 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.063991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424e752-ec34-4e59-bd77-1e8be1d8ed48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.064373 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4424e752-ec34-4e59-bd77-1e8be1d8ed48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:35.064793 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec6264-1f5c-4bce-bcea-4bb99104864f admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:35.074984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec6264-1f5c-4bce-bcea-4bb99104864f admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:35.079596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd2b35-80f2-4a9a-9c7c-76f24ab7b853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:35.080272 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd2b35-80f2-4a9a-9c7c-76f24ab7b853 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.080548 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd2b35-80f2-4a9a-9c7c-76f24ab7b853 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.081069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd2b35-80f2-4a9a-9c7c-76f24ab7b853 admin admin] PATH_INFO: `/v3/projects/94730f8e4762484e9bfcbbaf02cfd6ac/users/06575c72303c43b98dd75ff831626736/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.097033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd2b35-80f2-4a9a-9c7c-76f24ab7b853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94730f8e4762484e9bfcbbaf02cfd6ac, user_id=06575c72303c43b98dd75ff831626736, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:35.099713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd2b35-80f2-4a9a-9c7c-76f24ab7b853 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:35.129770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424e752-ec34-4e59-bd77-1e8be1d8ed48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:35.131266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424e752-ec34-4e59-bd77-1e8be1d8ed48 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:35.141498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1308/2619] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:50:35.180363 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1312/2620] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:50:35 2022] PUT /identity/v3/projects/94730f8e4762484e9bfcbbaf02cfd6ac/users/06575c72303c43b98dd75ff831626736/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:35.192034 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c649cf-de72-43a0-80d2-c4dab1b56f4d None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.192333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c649cf-de72-43a0-80d2-c4dab1b56f4d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.192616 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c649cf-de72-43a0-80d2-c4dab1b56f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.221660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-40c649cf-de72-43a0-80d2-c4dab1b56f4d None None] MFA Rules not processed for user `06575c72303c43b98dd75ff831626736`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:35.222941 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-40c649cf-de72-43a0-80d2-c4dab1b56f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:35.290866 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1309/2621] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:35 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:35.295323 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd2b35-80f2-4a9a-9c7c-76f24ab7b853 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:35.359013 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd2b35-80f2-4a9a-9c7c-76f24ab7b853 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:35.363817 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e7dc7-ecda-40b8-8601-4b091570d578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:35.364429 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e7dc7-ecda-40b8-8601-4b091570d578 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.364708 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e7dc7-ecda-40b8-8601-4b091570d578 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.364980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e7dc7-ecda-40b8-8601-4b091570d578 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.366187 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e7dc7-ecda-40b8-8601-4b091570d578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:35.368131 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e7dc7-ecda-40b8-8601-4b091570d578 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:35.467042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1313/2622] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:50:35 2022] POST /identity/v3/projects => generated 405 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:50:35.470993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c649cf-de72-43a0-80d2-c4dab1b56f4d None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:35.482836 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c649cf-de72-43a0-80d2-c4dab1b56f4d None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:35.529184 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:35.548423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c65ce3-746b-49f3-944f-7882328c7b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:35.548993 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c65ce3-746b-49f3-944f-7882328c7b17 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.549279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c65ce3-746b-49f3-944f-7882328c7b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.549564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c65ce3-746b-49f3-944f-7882328c7b17 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.550411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c65ce3-746b-49f3-944f-7882328c7b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:35.552358 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c65ce3-746b-49f3-944f-7882328c7b17 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:35.665769 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1310/2623] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:50:35 2022] POST /identity/v3/users => generated 309 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:50:35.670071 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e7dc7-ecda-40b8-8601-4b091570d578 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:35.675698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:35.681528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e7dc7-ecda-40b8-8601-4b091570d578 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:35.686619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b1ff5-a343-47bd-818a-5070f8f84df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:35.687229 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b1ff5-a343-47bd-818a-5070f8f84df7 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.687532 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b1ff5-a343-47bd-818a-5070f8f84df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.687803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b1ff5-a343-47bd-818a-5070f8f84df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.688622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b1ff5-a343-47bd-818a-5070f8f84df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:35.690277 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b1ff5-a343-47bd-818a-5070f8f84df7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:35.701338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1314/2624] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:50:35.705652 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c65ce3-746b-49f3-944f-7882328c7b17 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:35.716163 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c65ce3-746b-49f3-944f-7882328c7b17 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:35.721186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45919e1f-cc6f-411b-8fef-d857a921e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:35.721857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45919e1f-cc6f-411b-8fef-d857a921e16c admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.722167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45919e1f-cc6f-411b-8fef-d857a921e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.722586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45919e1f-cc6f-411b-8fef-d857a921e16c admin admin] PATH_INFO: `/v3/projects/1bf596ba4b684279b213e9686ea503a8/users/d7fe8a8c6e544afe8b77ad187a51f51d/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.745874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45919e1f-cc6f-411b-8fef-d857a921e16c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf596ba4b684279b213e9686ea503a8, user_id=d7fe8a8c6e544afe8b77ad187a51f51d, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:35.748949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45919e1f-cc6f-411b-8fef-d857a921e16c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:35.809082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1311/2625] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:50:35 2022] PUT /identity/v3/projects/1bf596ba4b684279b213e9686ea503a8/users/d7fe8a8c6e544afe8b77ad187a51f51d/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:35.813200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b1ff5-a343-47bd-818a-5070f8f84df7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:35.823783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b1ff5-a343-47bd-818a-5070f8f84df7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:35.881678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc305c-f929-40f2-aec0-4f36ed30d5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:35.882498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc305c-f929-40f2-aec0-4f36ed30d5ba admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.882778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc305c-f929-40f2-aec0-4f36ed30d5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.883061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc305c-f929-40f2-aec0-4f36ed30d5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.883810 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc305c-f929-40f2-aec0-4f36ed30d5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:35.885455 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc305c-f929-40f2-aec0-4f36ed30d5ba admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:35.895620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1315/2626] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:50:35.900568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45919e1f-cc6f-411b-8fef-d857a921e16c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:35.911015 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45919e1f-cc6f-411b-8fef-d857a921e16c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:35.916952 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889e24f-7bae-4e39-8494-7a8642ddfcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:35.917635 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889e24f-7bae-4e39-8494-7a8642ddfcc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.917924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889e24f-7bae-4e39-8494-7a8642ddfcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.918250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889e24f-7bae-4e39-8494-7a8642ddfcc5 admin admin] PATH_INFO: `/v3/projects/1bf596ba4b684279b213e9686ea503a8/users/d7fe8a8c6e544afe8b77ad187a51f51d/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.920235 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889e24f-7bae-4e39-8494-7a8642ddfcc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf596ba4b684279b213e9686ea503a8, user_id=d7fe8a8c6e544afe8b77ad187a51f51d, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:35.922922 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889e24f-7bae-4e39-8494-7a8642ddfcc5 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:35.933782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1312/2627] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:50:35 2022] PUT /identity/v3/projects/1bf596ba4b684279b213e9686ea503a8/users/d7fe8a8c6e544afe8b77ad187a51f51d/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:35.944824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ad8fe-9a93-4fa2-a1c8-118c5d4dccc4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:35.945157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ad8fe-9a93-4fa2-a1c8-118c5d4dccc4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:35.945443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ad8fe-9a93-4fa2-a1c8-118c5d4dccc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:35.975965 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-dd6ad8fe-9a93-4fa2-a1c8-118c5d4dccc4 None None] MFA Rules not processed for user `d7fe8a8c6e544afe8b77ad187a51f51d`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:35.977014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-dd6ad8fe-9a93-4fa2-a1c8-118c5d4dccc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:36.047445 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1316/2628] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:35 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:50:36.058938 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889e24f-7bae-4e39-8494-7a8642ddfcc5 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:36.060854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6889e24f-7bae-4e39-8494-7a8642ddfcc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:36.119279 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889e24f-7bae-4e39-8494-7a8642ddfcc5 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:36.123269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fae46-c426-4982-b95d-58fa37b5f17c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:36.123891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fae46-c426-4982-b95d-58fa37b5f17c service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:36.124179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fae46-c426-4982-b95d-58fa37b5f17c service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:36.124458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fae46-c426-4982-b95d-58fa37b5f17c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:36.134929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fae46-c426-4982-b95d-58fa37b5f17c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:36.136375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fae46-c426-4982-b95d-58fa37b5f17c service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:36.146044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1313/2629] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:36 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:50:36.298410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ad8fe-9a93-4fa2-a1c8-118c5d4dccc4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:36.366705 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ad8fe-9a93-4fa2-a1c8-118c5d4dccc4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:36.372093 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:36.372994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:36.373276 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:36.373551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] PATH_INFO: `/v3/users/06575c72303c43b98dd75ff831626736` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:36.375114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=06575c72303c43b98dd75ff831626736)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:36.377218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:36.391371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:36.570058 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06575c72303c43b98dd75ff831626736'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:36.619458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06575c72303c43b98dd75ff831626736'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:36.632635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06575c72303c43b98dd75ff831626736'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:36.633327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06575c72303c43b98dd75ff831626736'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:36.640744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06575c72303c43b98dd75ff831626736'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:36.646645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1317/2630] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:50:36 2022] DELETE /identity/v3/users/06575c72303c43b98dd75ff831626736 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:36.652032 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae0419-1c73-4fd0-9b9c-2ce91b0d70b9 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:36.652541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae0419-1c73-4fd0-9b9c-2ce91b0d70b9 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:36.652681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae0419-1c73-4fd0-9b9c-2ce91b0d70b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:36.672370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-88ae0419-1c73-4fd0-9b9c-2ce91b0d70b9 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:36.673585 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-88ae0419-1c73-4fd0-9b9c-2ce91b0d70b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:36.738466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1314/2631] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:36.750762 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:36.752227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:36.816830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3b24b-2908-4cf1-a295-e1a5d0b657ee admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:36.821032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dccf23-61dc-460e-8e1c-9d1d20a2be43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:36.821731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dccf23-61dc-460e-8e1c-9d1d20a2be43 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:36.822105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dccf23-61dc-460e-8e1c-9d1d20a2be43 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:36.822450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dccf23-61dc-460e-8e1c-9d1d20a2be43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:36.832147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dccf23-61dc-460e-8e1c-9d1d20a2be43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:36.833720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dccf23-61dc-460e-8e1c-9d1d20a2be43 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:36.844080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1318/2632] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:50:37.428492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae0419-1c73-4fd0-9b9c-2ce91b0d70b9 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:37.433772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-88ae0419-1c73-4fd0-9b9c-2ce91b0d70b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:37.444665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae0419-1c73-4fd0-9b9c-2ce91b0d70b9 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:37.455691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:37.456487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:37.460025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:37.460025 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] PATH_INFO: `/v3/projects/94730f8e4762484e9bfcbbaf02cfd6ac` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:37.460285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94730f8e4762484e9bfcbbaf02cfd6ac)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:37.472241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:37.575982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:37.597082 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94730f8e4762484e9bfcbbaf02cfd6ac', 'request_id': 'req-f0f6031c-26d4-4508-b94b-a07574758174'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:37.604280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94730f8e4762484e9bfcbbaf02cfd6ac', 'request_id': 'req-f0f6031c-26d4-4508-b94b-a07574758174'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:37.671790 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94730f8e4762484e9bfcbbaf02cfd6ac', 'request_id': 'req-f0f6031c-26d4-4508-b94b-a07574758174'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:37.683299 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] The token cache is being invalidate because project 94730f8e4762484e9bfcbbaf02cfd6ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:50:37.683604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:37.684272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:37.685956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1315/2633] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:50:37 2022] DELETE /identity/v3/projects/94730f8e4762484e9bfcbbaf02cfd6ac => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:37.690622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dccf23-61dc-460e-8e1c-9d1d20a2be43 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:37.703245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dccf23-61dc-460e-8e1c-9d1d20a2be43 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:37.765577 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fb535-c182-4d80-8e40-0c4668653cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:37.767161 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fb535-c182-4d80-8e40-0c4668653cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:37.767670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fb535-c182-4d80-8e40-0c4668653cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:37.768154 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fb535-c182-4d80-8e40-0c4668653cd7 admin admin] PATH_INFO: `/v3/users/d7fe8a8c6e544afe8b77ad187a51f51d` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:37.769738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fb535-c182-4d80-8e40-0c4668653cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7fe8a8c6e544afe8b77ad187a51f51d)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:37.775054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fb535-c182-4d80-8e40-0c4668653cd7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:37.952295 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:37.962984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-598fb535-c182-4d80-8e40-0c4668653cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7fe8a8c6e544afe8b77ad187a51f51d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:38.059711 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-598fb535-c182-4d80-8e40-0c4668653cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7fe8a8c6e544afe8b77ad187a51f51d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:38.075882 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-598fb535-c182-4d80-8e40-0c4668653cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7fe8a8c6e544afe8b77ad187a51f51d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:38.077148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-598fb535-c182-4d80-8e40-0c4668653cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7fe8a8c6e544afe8b77ad187a51f51d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:38.085802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-598fb535-c182-4d80-8e40-0c4668653cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7fe8a8c6e544afe8b77ad187a51f51d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:38.100619 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1319/2634] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:50:37 2022] DELETE /identity/v3/users/d7fe8a8c6e544afe8b77ad187a51f51d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:38.593642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:38.595254 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:38.611134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6031c-26d4-4508-b94b-a07574758174 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:38.676879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:38.677610 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:38.677799 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:38.678145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] PATH_INFO: `/v3/projects/1bf596ba4b684279b213e9686ea503a8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:38.679968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf596ba4b684279b213e9686ea503a8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:38.681986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:39.004950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf596ba4b684279b213e9686ea503a8', 'request_id': 'req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:39.012661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf596ba4b684279b213e9686ea503a8', 'request_id': 'req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:39.076260 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf596ba4b684279b213e9686ea503a8', 'request_id': 'req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:39.090812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] The token cache is being invalidate because project 1bf596ba4b684279b213e9686ea503a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:50:39.091109 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:39.091789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4d3eb2f0-a53e-4e4e-97a0-cc0629122017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:39.093364 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1316/2635] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:50:38 2022] DELETE /identity/v3/projects/1bf596ba4b684279b213e9686ea503a8 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:39.098508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0bd46-440e-4223-96ee-3ffa1009319c None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:39.099019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0bd46-440e-4223-96ee-3ffa1009319c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:39.099148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0bd46-440e-4223-96ee-3ffa1009319c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:39.118821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-01f0bd46-440e-4223-96ee-3ffa1009319c None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:39.120057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-01f0bd46-440e-4223-96ee-3ffa1009319c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:39.187327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1320/2636] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:39.196185 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b0fb-528f-44a1-bb8a-9f71543fbb64 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:39.196474 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b0fb-528f-44a1-bb8a-9f71543fbb64 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:39.196854 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b0fb-528f-44a1-bb8a-9f71543fbb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:39.217628 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-0475b0fb-528f-44a1-bb8a-9f71543fbb64 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:39.218880 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-0475b0fb-528f-44a1-bb8a-9f71543fbb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:39.227121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1317/2637] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:50:39.231573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0bd46-440e-4223-96ee-3ffa1009319c None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:39.241784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0bd46-440e-4223-96ee-3ffa1009319c None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:39.246552 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60206617-5461-4ca2-8a5e-bc786896e573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:39.247164 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60206617-5461-4ca2-8a5e-bc786896e573 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:39.247704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60206617-5461-4ca2-8a5e-bc786896e573 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:39.247979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60206617-5461-4ca2-8a5e-bc786896e573 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:39.248672 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60206617-5461-4ca2-8a5e-bc786896e573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:50:39.248981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60206617-5461-4ca2-8a5e-bc786896e573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:39.250607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60206617-5461-4ca2-8a5e-bc786896e573 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:39.263856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1321/2638] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:50:39.267920 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475b0fb-528f-44a1-bb8a-9f71543fbb64 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:39.278734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475b0fb-528f-44a1-bb8a-9f71543fbb64 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:39.283702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001d6c2-42e9-4ebc-9834-7dafd78d88fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:39.284365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001d6c2-42e9-4ebc-9834-7dafd78d88fc admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:39.284660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001d6c2-42e9-4ebc-9834-7dafd78d88fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:39.284927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001d6c2-42e9-4ebc-9834-7dafd78d88fc admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:39.285608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001d6c2-42e9-4ebc-9834-7dafd78d88fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:39.287600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001d6c2-42e9-4ebc-9834-7dafd78d88fc admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:39.387817 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1318/2639] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:50:39 2022] POST /identity/v3/projects => generated 399 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:50:39.392060 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60206617-5461-4ca2-8a5e-bc786896e573 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:39.403136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60206617-5461-4ca2-8a5e-bc786896e573 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:39.469685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cfd51-ae44-4f14-92ca-b31703ce0ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:39.470352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cfd51-ae44-4f14-92ca-b31703ce0ffb admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:39.470638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cfd51-ae44-4f14-92ca-b31703ce0ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:39.470913 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cfd51-ae44-4f14-92ca-b31703ce0ffb admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:39.471579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cfd51-ae44-4f14-92ca-b31703ce0ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:39.473469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cfd51-ae44-4f14-92ca-b31703ce0ffb admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:39.554040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1322/2640] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:50:39 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:50:39.555981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001d6c2-42e9-4ebc-9834-7dafd78d88fc admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:39.566342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001d6c2-42e9-4ebc-9834-7dafd78d88fc admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:39.571733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1f0a3-2440-4c0a-ba26-8c384eaf6d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:39.572350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1f0a3-2440-4c0a-ba26-8c384eaf6d4a admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:39.572632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1f0a3-2440-4c0a-ba26-8c384eaf6d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:39.572911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1f0a3-2440-4c0a-ba26-8c384eaf6d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:39.573602 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1f0a3-2440-4c0a-ba26-8c384eaf6d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:39.575396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1f0a3-2440-4c0a-ba26-8c384eaf6d4a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:39.587553 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1319/2641] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:50:39.591856 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cfd51-ae44-4f14-92ca-b31703ce0ffb admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:39.603440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cfd51-ae44-4f14-92ca-b31703ce0ffb admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:39.610173 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b311d5-2dd4-43b1-9afe-2bdf8f0bbdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:39.610810 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b311d5-2dd4-43b1-9afe-2bdf8f0bbdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:39.611096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b311d5-2dd4-43b1-9afe-2bdf8f0bbdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:39.611371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b311d5-2dd4-43b1-9afe-2bdf8f0bbdf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:39.612104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b311d5-2dd4-43b1-9afe-2bdf8f0bbdf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:39.613714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b311d5-2dd4-43b1-9afe-2bdf8f0bbdf0 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:39.626111 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1323/2642] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:50:39.630612 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1f0a3-2440-4c0a-ba26-8c384eaf6d4a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:39.637605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:39.641547 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1f0a3-2440-4c0a-ba26-8c384eaf6d4a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:39.646378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b34a4-896f-470a-9c19-629eea6508fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:39.649600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b34a4-896f-470a-9c19-629eea6508fc admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:39.649600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b34a4-896f-470a-9c19-629eea6508fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:39.649600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b34a4-896f-470a-9c19-629eea6508fc admin admin] PATH_INFO: `/v3/projects/1d5ccf1a07b44e398308e66c3a439d7d/users/10a9800290fa4cb9898bcae991ef6765/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:39.666609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b34a4-896f-470a-9c19-629eea6508fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d5ccf1a07b44e398308e66c3a439d7d, user_id=10a9800290fa4cb9898bcae991ef6765, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:39.669295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b34a4-896f-470a-9c19-629eea6508fc admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:39.780569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1320/2643] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:50:39 2022] PUT /identity/v3/projects/1d5ccf1a07b44e398308e66c3a439d7d/users/10a9800290fa4cb9898bcae991ef6765/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:39.792474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c6356-522f-4512-a715-45cce14c32db None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:39.792892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c6356-522f-4512-a715-45cce14c32db None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:39.793081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c6356-522f-4512-a715-45cce14c32db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:39.821227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-c40c6356-522f-4512-a715-45cce14c32db None None] MFA Rules not processed for user `10a9800290fa4cb9898bcae991ef6765`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:39.822535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c40c6356-522f-4512-a715-45cce14c32db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:39.830598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:39.891149 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1324/2644] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:39 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:39.892540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b34a4-896f-470a-9c19-629eea6508fc admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:39.967248 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b34a4-896f-470a-9c19-629eea6508fc admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:39.972246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7a596-397b-4f44-8668-d261f9303a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:39.972708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7a596-397b-4f44-8668-d261f9303a55 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:39.972998 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7a596-397b-4f44-8668-d261f9303a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:39.973276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7a596-397b-4f44-8668-d261f9303a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:39.973946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7a596-397b-4f44-8668-d261f9303a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:39.976164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7a596-397b-4f44-8668-d261f9303a55 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:40.095202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1321/2645] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:50:39 2022] POST /identity/v3/projects => generated 399 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:50:40.099937 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c6356-522f-4512-a715-45cce14c32db None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:40.112678 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c6356-522f-4512-a715-45cce14c32db None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:40.176764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb9941-f98d-41f8-b19c-2891985e51f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:40.177387 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb9941-f98d-41f8-b19c-2891985e51f2 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:40.177708 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb9941-f98d-41f8-b19c-2891985e51f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:40.178642 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb9941-f98d-41f8-b19c-2891985e51f2 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:40.179503 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb9941-f98d-41f8-b19c-2891985e51f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:40.181583 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb9941-f98d-41f8-b19c-2891985e51f2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:40.263012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1325/2646] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:50:40 2022] POST /identity/v3/users => generated 306 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:50:40.264319 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7a596-397b-4f44-8668-d261f9303a55 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:40.275837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7a596-397b-4f44-8668-d261f9303a55 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:40.283554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33e10b-694e-4ca1-bc58-155a90bb232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:40.283554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33e10b-694e-4ca1-bc58-155a90bb232d admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:40.283554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33e10b-694e-4ca1-bc58-155a90bb232d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:40.283554 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33e10b-694e-4ca1-bc58-155a90bb232d admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:40.283985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33e10b-694e-4ca1-bc58-155a90bb232d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:40.284272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33e10b-694e-4ca1-bc58-155a90bb232d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:40.298497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1322/2647] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:50:40.302881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb9941-f98d-41f8-b19c-2891985e51f2 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:40.313308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb9941-f98d-41f8-b19c-2891985e51f2 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:40.319613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5ebbf-c86d-48ff-a578-cc4260eb50e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:40.319613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ebbf-c86d-48ff-a578-cc4260eb50e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:40.319613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ebbf-c86d-48ff-a578-cc4260eb50e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:40.319984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ebbf-c86d-48ff-a578-cc4260eb50e3 admin admin] PATH_INFO: `/v3/projects/d2b0900c774843e3b53c60ba4033b7e7/users/5cdb95c529194a6a88c2c2703332a37b/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:40.336118 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5ebbf-c86d-48ff-a578-cc4260eb50e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b0900c774843e3b53c60ba4033b7e7, user_id=5cdb95c529194a6a88c2c2703332a37b, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:40.339364 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5ebbf-c86d-48ff-a578-cc4260eb50e3 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:40.421057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1326/2648] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:50:40 2022] PUT /identity/v3/projects/d2b0900c774843e3b53c60ba4033b7e7/users/5cdb95c529194a6a88c2c2703332a37b/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:50:40.425115 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33e10b-694e-4ca1-bc58-155a90bb232d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:40.435135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33e10b-694e-4ca1-bc58-155a90bb232d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:40.501969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5b5fd-4038-47ee-a9ae-ddd80124f055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:40.502683 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b5fd-4038-47ee-a9ae-ddd80124f055 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:40.502968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b5fd-4038-47ee-a9ae-ddd80124f055 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:40.503278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b5fd-4038-47ee-a9ae-ddd80124f055 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:40.504018 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5b5fd-4038-47ee-a9ae-ddd80124f055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:40.505690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5b5fd-4038-47ee-a9ae-ddd80124f055 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:40.529013 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1323/2649] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:50:40.529854 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5ebbf-c86d-48ff-a578-cc4260eb50e3 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:40.549695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5ebbf-c86d-48ff-a578-cc4260eb50e3 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:40.551185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b7f1c-84a0-4a1b-8e9b-cf4e73864cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:40.551962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b7f1c-84a0-4a1b-8e9b-cf4e73864cda admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:40.552312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b7f1c-84a0-4a1b-8e9b-cf4e73864cda admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:40.552651 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b7f1c-84a0-4a1b-8e9b-cf4e73864cda admin admin] PATH_INFO: `/v3/projects/d2b0900c774843e3b53c60ba4033b7e7/users/5cdb95c529194a6a88c2c2703332a37b/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:40.555383 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b7f1c-84a0-4a1b-8e9b-cf4e73864cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b0900c774843e3b53c60ba4033b7e7, user_id=5cdb95c529194a6a88c2c2703332a37b, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:40.558462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b7f1c-84a0-4a1b-8e9b-cf4e73864cda admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:40.574758 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1327/2650] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:50:40 2022] PUT /identity/v3/projects/d2b0900c774843e3b53c60ba4033b7e7/users/5cdb95c529194a6a88c2c2703332a37b/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:40.585759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ec538-71b5-48fa-b5c3-fb95aaceeb49 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:40.586158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ec538-71b5-48fa-b5c3-fb95aaceeb49 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:40.586397 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ec538-71b5-48fa-b5c3-fb95aaceeb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:40.618713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-546ec538-71b5-48fa-b5c3-fb95aaceeb49 None None] MFA Rules not processed for user `5cdb95c529194a6a88c2c2703332a37b`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:40.620083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-546ec538-71b5-48fa-b5c3-fb95aaceeb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:40.667140 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:40.702770 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1324/2651] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:40 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:40.714493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b7f1c-84a0-4a1b-8e9b-cf4e73864cda admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:40.718110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-835b7f1c-84a0-4a1b-8e9b-cf4e73864cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:40.798144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b7f1c-84a0-4a1b-8e9b-cf4e73864cda admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:40.802761 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae85a5-e040-473e-bb4a-984db08e91d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:40.802761 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae85a5-e040-473e-bb4a-984db08e91d9 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:40.803623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae85a5-e040-473e-bb4a-984db08e91d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:40.804233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae85a5-e040-473e-bb4a-984db08e91d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:40.818206 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae85a5-e040-473e-bb4a-984db08e91d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:40.821803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae85a5-e040-473e-bb4a-984db08e91d9 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:40.832021 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1328/2652] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:40 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:50:41.098783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ec538-71b5-48fa-b5c3-fb95aaceeb49 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:41.109549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ec538-71b5-48fa-b5c3-fb95aaceeb49 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:41.113189 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910aa53f-c017-49b6-a292-e26a0b403f96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:41.113837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910aa53f-c017-49b6-a292-e26a0b403f96 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:41.114193 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910aa53f-c017-49b6-a292-e26a0b403f96 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:41.114526 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910aa53f-c017-49b6-a292-e26a0b403f96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:41.179728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910aa53f-c017-49b6-a292-e26a0b403f96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:41.181360 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910aa53f-c017-49b6-a292-e26a0b403f96 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:41.191250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1325/2653] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:41 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:50:47.457382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae85a5-e040-473e-bb4a-984db08e91d9 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:47.462750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:47.522981 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae85a5-e040-473e-bb4a-984db08e91d9 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:47.528660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:47.529303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:47.529580 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:47.529850 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] PATH_INFO: `/v3/users/10a9800290fa4cb9898bcae991ef6765` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:47.531041 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a9800290fa4cb9898bcae991ef6765)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:47.533168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:47.724632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a9800290fa4cb9898bcae991ef6765'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:47.795957 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a9800290fa4cb9898bcae991ef6765'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:47.808685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a9800290fa4cb9898bcae991ef6765'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:47.809348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a9800290fa4cb9898bcae991ef6765'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:47.816219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a9800290fa4cb9898bcae991ef6765'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:47.822092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1329/2654] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:50:47 2022] DELETE /identity/v3/users/10a9800290fa4cb9898bcae991ef6765 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:47.827211 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0f362-29bc-4726-8e1b-f75a031d0d22 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:47.827731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0f362-29bc-4726-8e1b-f75a031d0d22 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:47.827837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0f362-29bc-4726-8e1b-f75a031d0d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:47.848098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ebc0f362-29bc-4726-8e1b-f75a031d0d22 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:47.849325 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ebc0f362-29bc-4726-8e1b-f75a031d0d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:47.909013 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1326/2655] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:47.919210 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:47.920601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:47.979853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fc225-efb2-4bec-8622-47fb4ee18968 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:47.984378 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3336d-3567-4562-8e97-bc1e058d2545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:47.985001 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3336d-3567-4562-8e97-bc1e058d2545 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:47.985276 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3336d-3567-4562-8e97-bc1e058d2545 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:47.985544 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3336d-3567-4562-8e97-bc1e058d2545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:47.995219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3336d-3567-4562-8e97-bc1e058d2545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:47.996725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3336d-3567-4562-8e97-bc1e058d2545 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:48.006473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1330/2656] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:50:48.735052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0f362-29bc-4726-8e1b-f75a031d0d22 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:48.736648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ebc0f362-29bc-4726-8e1b-f75a031d0d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:48.749092 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0f362-29bc-4726-8e1b-f75a031d0d22 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:48.754159 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:48.755863 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:48.755863 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:48.755863 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] PATH_INFO: `/v3/projects/1d5ccf1a07b44e398308e66c3a439d7d` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:48.756923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d5ccf1a07b44e398308e66c3a439d7d)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:48.758986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:48.864983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d5ccf1a07b44e398308e66c3a439d7d', 'request_id': 'req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:48.871495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d5ccf1a07b44e398308e66c3a439d7d', 'request_id': 'req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:48.921688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d5ccf1a07b44e398308e66c3a439d7d', 'request_id': 'req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:48.929280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:48.934221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] The token cache is being invalidate because project 1d5ccf1a07b44e398308e66c3a439d7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:50:48.934473 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:48.935133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:48.936665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1327/2657] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:50:48 2022] DELETE /identity/v3/projects/1d5ccf1a07b44e398308e66c3a439d7d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:48.940623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3336d-3567-4562-8e97-bc1e058d2545 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:48.951374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3336d-3567-4562-8e97-bc1e058d2545 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:48.975081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:49.007864 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e923f2-0196-459b-af85-36c16ba4c778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:49.008428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e923f2-0196-459b-af85-36c16ba4c778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:49.008707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e923f2-0196-459b-af85-36c16ba4c778 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:49.008979 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e923f2-0196-459b-af85-36c16ba4c778 admin admin] PATH_INFO: `/v3/users/5cdb95c529194a6a88c2c2703332a37b` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:49.010813 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e923f2-0196-459b-af85-36c16ba4c778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cdb95c529194a6a88c2c2703332a37b)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:49.012845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e923f2-0196-459b-af85-36c16ba4c778 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:49.210818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b6e923f2-0196-459b-af85-36c16ba4c778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cdb95c529194a6a88c2c2703332a37b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:49.304526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b6e923f2-0196-459b-af85-36c16ba4c778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cdb95c529194a6a88c2c2703332a37b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:49.317235 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b6e923f2-0196-459b-af85-36c16ba4c778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cdb95c529194a6a88c2c2703332a37b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:49.323314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b6e923f2-0196-459b-af85-36c16ba4c778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cdb95c529194a6a88c2c2703332a37b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:49.325865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-b6e923f2-0196-459b-af85-36c16ba4c778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cdb95c529194a6a88c2c2703332a37b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:49.332011 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1331/2658] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:50:48 2022] DELETE /identity/v3/users/5cdb95c529194a6a88c2c2703332a37b => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:49.802757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:49.804292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:49.817509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50be0f0-47f4-49d7-b99d-15bf5ea11c64 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:49.875318 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:49.876031 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:49.876233 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:49.876512 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] PATH_INFO: `/v3/projects/d2b0900c774843e3b53c60ba4033b7e7` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:49.877666 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b0900c774843e3b53c60ba4033b7e7)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:49.879646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:50.031290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b0900c774843e3b53c60ba4033b7e7', 'request_id': 'req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:50.037729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b0900c774843e3b53c60ba4033b7e7', 'request_id': 'req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:50.116448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b0900c774843e3b53c60ba4033b7e7', 'request_id': 'req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:50.128084 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] The token cache is being invalidate because project d2b0900c774843e3b53c60ba4033b7e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:50:50.128378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:50.129008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-0b04a6ac-822a-4bad-a2b8-df2ad84860d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:50:50.130563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1328/2659] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:50:49 2022] DELETE /identity/v3/projects/d2b0900c774843e3b53c60ba4033b7e7 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:50:50.136296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52614b8b-f2de-4159-8ffe-703b53e338de None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:50.136824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52614b8b-f2de-4159-8ffe-703b53e338de None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:50.136824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52614b8b-f2de-4159-8ffe-703b53e338de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:50.157457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-52614b8b-f2de-4159-8ffe-703b53e338de None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:50.158825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-52614b8b-f2de-4159-8ffe-703b53e338de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:50.217785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1332/2660] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:50.225820 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df2c54-47ee-4d51-9101-53c3e683abad None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:50.228537 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df2c54-47ee-4d51-9101-53c3e683abad None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:50.228537 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df2c54-47ee-4d51-9101-53c3e683abad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:50.247851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-40df2c54-47ee-4d51-9101-53c3e683abad None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:50.249043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-40df2c54-47ee-4d51-9101-53c3e683abad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:50.257047 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1329/2661] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:50.262269 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52614b8b-f2de-4159-8ffe-703b53e338de None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:50.272075 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52614b8b-f2de-4159-8ffe-703b53e338de None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:50.277324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f56d89-b7bd-4455-8f86-2d4c0cd668a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:50.277932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f56d89-b7bd-4455-8f86-2d4c0cd668a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:50.278268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f56d89-b7bd-4455-8f86-2d4c0cd668a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:50.278543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f56d89-b7bd-4455-8f86-2d4c0cd668a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:50.279302 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f56d89-b7bd-4455-8f86-2d4c0cd668a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:50:50.279627 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f56d89-b7bd-4455-8f86-2d4c0cd668a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:50.281214 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f56d89-b7bd-4455-8f86-2d4c0cd668a9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:50.293062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1333/2662] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:50:50.297222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df2c54-47ee-4d51-9101-53c3e683abad None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:50.306496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df2c54-47ee-4d51-9101-53c3e683abad None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:50.311487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216716ce-8655-4f0c-8165-36fe9133b1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:50.312135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216716ce-8655-4f0c-8165-36fe9133b1a2 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:50.312420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216716ce-8655-4f0c-8165-36fe9133b1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:50.312691 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216716ce-8655-4f0c-8165-36fe9133b1a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:50.313375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216716ce-8655-4f0c-8165-36fe9133b1a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:50.315310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216716ce-8655-4f0c-8165-36fe9133b1a2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:50.414324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1330/2663] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:50:50 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:50:50.418704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f56d89-b7bd-4455-8f86-2d4c0cd668a9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:50.428754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f56d89-b7bd-4455-8f86-2d4c0cd668a9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:50.460535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:50.486165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64ce6b-e160-4cff-9ae4-2c30a5ea9e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:50.486770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64ce6b-e160-4cff-9ae4-2c30a5ea9e61 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:50.487046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64ce6b-e160-4cff-9ae4-2c30a5ea9e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:50.487313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64ce6b-e160-4cff-9ae4-2c30a5ea9e61 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:50.487967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64ce6b-e160-4cff-9ae4-2c30a5ea9e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:50.489854 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64ce6b-e160-4cff-9ae4-2c30a5ea9e61 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:50.619074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1334/2664] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:50:50 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:50:50.623981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216716ce-8655-4f0c-8165-36fe9133b1a2 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:50.634142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216716ce-8655-4f0c-8165-36fe9133b1a2 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:50.639447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2c0aa-95be-4a56-934b-7de412d3ecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:50.640049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2c0aa-95be-4a56-934b-7de412d3ecdb admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:50.640329 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2c0aa-95be-4a56-934b-7de412d3ecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:50.640651 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2c0aa-95be-4a56-934b-7de412d3ecdb admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:50.641329 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2c0aa-95be-4a56-934b-7de412d3ecdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:50.642948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2c0aa-95be-4a56-934b-7de412d3ecdb admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:50.779859 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1331/2665] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:50:50.783947 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64ce6b-e160-4cff-9ae4-2c30a5ea9e61 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:50.794677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64ce6b-e160-4cff-9ae4-2c30a5ea9e61 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:50.799300 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cc947-7058-4836-a980-20b4b9559fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:50.799926 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cc947-7058-4836-a980-20b4b9559fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:50.800205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cc947-7058-4836-a980-20b4b9559fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:50.800481 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cc947-7058-4836-a980-20b4b9559fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:50.801167 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cc947-7058-4836-a980-20b4b9559fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:50.802795 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cc947-7058-4836-a980-20b4b9559fd4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:50.813576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1335/2666] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:50:50.817822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2c0aa-95be-4a56-934b-7de412d3ecdb admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:50.828002 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2c0aa-95be-4a56-934b-7de412d3ecdb admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:50.832470 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba311f-a241-4f87-a0b4-aa3e304453a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:50.833144 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba311f-a241-4f87-a0b4-aa3e304453a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:50.833419 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba311f-a241-4f87-a0b4-aa3e304453a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:50.833686 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba311f-a241-4f87-a0b4-aa3e304453a5 admin admin] PATH_INFO: `/v3/projects/e92b9ac0d0c94731b078faf1255d9b13/users/60cfe71b7f0a4fa49f2754a8e4ce7413/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:50.847914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba311f-a241-4f87-a0b4-aa3e304453a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e92b9ac0d0c94731b078faf1255d9b13, user_id=60cfe71b7f0a4fa49f2754a8e4ce7413, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:50.850556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba311f-a241-4f87-a0b4-aa3e304453a5 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:50.939342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1332/2667] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:50:50 2022] PUT /identity/v3/projects/e92b9ac0d0c94731b078faf1255d9b13/users/60cfe71b7f0a4fa49f2754a8e4ce7413/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:50:50.949839 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfce876-b201-495c-971c-b5ec70f45613 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:50.950169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfce876-b201-495c-971c-b5ec70f45613 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:50.950450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfce876-b201-495c-971c-b5ec70f45613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:50.978448 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-5dfce876-b201-495c-971c-b5ec70f45613 None None] MFA Rules not processed for user `60cfe71b7f0a4fa49f2754a8e4ce7413`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:50:50.979663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-5dfce876-b201-495c-971c-b5ec70f45613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:51.044316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1336/2668] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:50:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:50:51.055120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba311f-a241-4f87-a0b4-aa3e304453a5 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:50:51.056819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-7bba311f-a241-4f87-a0b4-aa3e304453a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:50:51.084490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:50:51.114841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba311f-a241-4f87-a0b4-aa3e304453a5 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:50:51.118541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2be2c1-fb21-4f2c-bec1-927578c3e429 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a9a6bcfa1b469298774e09a8912b93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:50:51.119158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2be2c1-fb21-4f2c-bec1-927578c3e429 service glance] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:50:51.119440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2be2c1-fb21-4f2c-bec1-927578c3e429 service glance] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:50:51.119710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2be2c1-fb21-4f2c-bec1-927578c3e429 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:50:51.129228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2be2c1-fb21-4f2c-bec1-927578c3e429 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:50:51.130689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2be2c1-fb21-4f2c-bec1-927578c3e429 service glance] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:50:51.139857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1333/2669] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:50:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:51:01.040337 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:01.212815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=15 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:17.851172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfce876-b201-495c-971c-b5ec70f45613 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:17.914238 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfce876-b201-495c-971c-b5ec70f45613 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:17.918799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:17.919435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:17.919720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:17.920022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] PATH_INFO: `/v3/users/60cfe71b7f0a4fa49f2754a8e4ce7413` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:17.921176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60cfe71b7f0a4fa49f2754a8e4ce7413)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:17.923136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:18.145840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60cfe71b7f0a4fa49f2754a8e4ce7413'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:18.235288 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60cfe71b7f0a4fa49f2754a8e4ce7413'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:18.249659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60cfe71b7f0a4fa49f2754a8e4ce7413'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:18.253472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60cfe71b7f0a4fa49f2754a8e4ce7413'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:18.257242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60cfe71b7f0a4fa49f2754a8e4ce7413'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:18.263208 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1337/2670] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:51:17 2022] DELETE /identity/v3/users/60cfe71b7f0a4fa49f2754a8e4ce7413 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:51:18.267681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c553-42b6-42aa-a55b-2dbf7099431a None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:18.268178 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c553-42b6-42aa-a55b-2dbf7099431a None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:18.268294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c553-42b6-42aa-a55b-2dbf7099431a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:18.287446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-e677c553-42b6-42aa-a55b-2dbf7099431a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:18.289207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e677c553-42b6-42aa-a55b-2dbf7099431a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:18.348699 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1334/2671] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:18.359304 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:18.362296 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:18.418624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5a60d-ba72-4c14-baf5-788d648fb2f6 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:18.422110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c51639-2456-4e8b-b512-c3182d571cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:18.422804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c51639-2456-4e8b-b512-c3182d571cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:18.423084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c51639-2456-4e8b-b512-c3182d571cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:18.423352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c51639-2456-4e8b-b512-c3182d571cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:18.432419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c51639-2456-4e8b-b512-c3182d571cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:18.433909 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c51639-2456-4e8b-b512-c3182d571cc2 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:18.443410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1338/2672] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:51:18.878084 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677c553-42b6-42aa-a55b-2dbf7099431a None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:18.883300 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e677c553-42b6-42aa-a55b-2dbf7099431a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:18.893684 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677c553-42b6-42aa-a55b-2dbf7099431a None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:18.898905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:18.899868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:18.900707 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:18.900992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] PATH_INFO: `/v3/projects/e92b9ac0d0c94731b078faf1255d9b13` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:18.902368 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e92b9ac0d0c94731b078faf1255d9b13)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:18.904261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:18.989814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e92b9ac0d0c94731b078faf1255d9b13', 'request_id': 'req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:18.996266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e92b9ac0d0c94731b078faf1255d9b13', 'request_id': 'req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:19.058211 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e92b9ac0d0c94731b078faf1255d9b13', 'request_id': 'req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:19.069618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] The token cache is being invalidate because project e92b9ac0d0c94731b078faf1255d9b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:51:19.069902 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:19.070712 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8f4c0755-9711-4c17-ba73-8d6c01dbd2bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:19.072199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1335/2673] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:51:18 2022] DELETE /identity/v3/projects/e92b9ac0d0c94731b078faf1255d9b13 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:51:19.077064 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5eb7b-b6d4-47ca-a996-b3623cfd728a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:19.077542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5eb7b-b6d4-47ca-a996-b3623cfd728a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:19.077641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5eb7b-b6d4-47ca-a996-b3623cfd728a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:19.097718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-2bb5eb7b-b6d4-47ca-a996-b3623cfd728a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:19.098943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2bb5eb7b-b6d4-47ca-a996-b3623cfd728a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:19.116324 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:19.160310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1339/2674] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:19.168166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21620e19-a971-4098-8347-3a8e9287b17b None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:19.168458 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21620e19-a971-4098-8347-3a8e9287b17b None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:19.168739 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21620e19-a971-4098-8347-3a8e9287b17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:19.188112 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-21620e19-a971-4098-8347-3a8e9287b17b None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:19.190639 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-21620e19-a971-4098-8347-3a8e9287b17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:19.197959 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1336/2675] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:19.202367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5eb7b-b6d4-47ca-a996-b3623cfd728a None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:19.213316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5eb7b-b6d4-47ca-a996-b3623cfd728a None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:19.217846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9f6f0-c62e-4f13-8ee3-1bb91e185e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:19.233624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9f6f0-c62e-4f13-8ee3-1bb91e185e1a admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:19.233624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9f6f0-c62e-4f13-8ee3-1bb91e185e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:19.233624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9f6f0-c62e-4f13-8ee3-1bb91e185e1a admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:19.233624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9f6f0-c62e-4f13-8ee3-1bb91e185e1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:51:19.233624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9f6f0-c62e-4f13-8ee3-1bb91e185e1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:19.233624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9f6f0-c62e-4f13-8ee3-1bb91e185e1a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:19.234586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1340/2676] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:51:19.238538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21620e19-a971-4098-8347-3a8e9287b17b None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:19.248733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21620e19-a971-4098-8347-3a8e9287b17b None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:19.253343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1d952-647d-409b-bcb9-da332693726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:19.253950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1d952-647d-409b-bcb9-da332693726d admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:19.254273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1d952-647d-409b-bcb9-da332693726d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:19.254546 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1d952-647d-409b-bcb9-da332693726d admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:19.255199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1d952-647d-409b-bcb9-da332693726d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:19.257120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1d952-647d-409b-bcb9-da332693726d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:19.403764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1337/2677] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:51:19 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:51:19.408012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9f6f0-c62e-4f13-8ee3-1bb91e185e1a admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:19.418263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9f6f0-c62e-4f13-8ee3-1bb91e185e1a admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:19.474536 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eed9d9-f40f-473c-9796-6d26a0f2c8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:19.475150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eed9d9-f40f-473c-9796-6d26a0f2c8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:19.475437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eed9d9-f40f-473c-9796-6d26a0f2c8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:19.475714 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eed9d9-f40f-473c-9796-6d26a0f2c8d6 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:19.476498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eed9d9-f40f-473c-9796-6d26a0f2c8d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:19.478764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eed9d9-f40f-473c-9796-6d26a0f2c8d6 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:19.584184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1341/2678] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:51:19 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:51:19.588373 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1d952-647d-409b-bcb9-da332693726d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:19.598684 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1d952-647d-409b-bcb9-da332693726d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:19.603682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05fe52-5f74-4f08-91bc-26469a4ce320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:19.604284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05fe52-5f74-4f08-91bc-26469a4ce320 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:19.604567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05fe52-5f74-4f08-91bc-26469a4ce320 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:19.604841 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05fe52-5f74-4f08-91bc-26469a4ce320 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:19.605606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05fe52-5f74-4f08-91bc-26469a4ce320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:19.607653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05fe52-5f74-4f08-91bc-26469a4ce320 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:19.613080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eed9d9-f40f-473c-9796-6d26a0f2c8d6 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:19.614746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-27eed9d9-f40f-473c-9796-6d26a0f2c8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:19.619636 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1338/2679] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:51:19.623876 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05fe52-5f74-4f08-91bc-26469a4ce320 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:19.633794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05fe52-5f74-4f08-91bc-26469a4ce320 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:19.639059 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31398b4-a55f-4e2a-ae36-853513fc37fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:19.639665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31398b4-a55f-4e2a-ae36-853513fc37fa admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:19.639985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31398b4-a55f-4e2a-ae36-853513fc37fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:19.640283 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31398b4-a55f-4e2a-ae36-853513fc37fa admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:19.640969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31398b4-a55f-4e2a-ae36-853513fc37fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:19.642623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31398b4-a55f-4e2a-ae36-853513fc37fa admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:19.652741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1339/2680] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:51:19.656868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31398b4-a55f-4e2a-ae36-853513fc37fa admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:19.666317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31398b4-a55f-4e2a-ae36-853513fc37fa admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:19.671008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59293b-484e-4145-b7d1-18cde7915a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:19.671653 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59293b-484e-4145-b7d1-18cde7915a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:19.671937 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59293b-484e-4145-b7d1-18cde7915a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:19.672217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59293b-484e-4145-b7d1-18cde7915a5e admin admin] PATH_INFO: `/v3/projects/4bc3f55f81ae45a8906d2349f694ef53/users/1beceb4bb1f24552af7bb04c73b97464/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:19.685401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eed9d9-f40f-473c-9796-6d26a0f2c8d6 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:19.688295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59293b-484e-4145-b7d1-18cde7915a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc3f55f81ae45a8906d2349f694ef53, user_id=1beceb4bb1f24552af7bb04c73b97464, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:19.689148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc873081-43a3-49b3-a43e-b71e4e0c0534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:19.689764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc873081-43a3-49b3-a43e-b71e4e0c0534 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:19.690073 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc873081-43a3-49b3-a43e-b71e4e0c0534 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:19.690389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc873081-43a3-49b3-a43e-b71e4e0c0534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:19.690954 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59293b-484e-4145-b7d1-18cde7915a5e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:19.692202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-fc873081-43a3-49b3-a43e-b71e4e0c0534 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:19.747027 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:19.775034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc873081-43a3-49b3-a43e-b71e4e0c0534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:19.775034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc873081-43a3-49b3-a43e-b71e4e0c0534 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:19.778152 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1340/2681] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:51:19 2022] PUT /identity/v3/projects/4bc3f55f81ae45a8906d2349f694ef53/users/1beceb4bb1f24552af7bb04c73b97464/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:51:19.790052 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5ce2c-2daa-4eeb-a0b8-cbd44355537d None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:19.790373 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5ce2c-2daa-4eeb-a0b8-cbd44355537d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:19.790650 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5ce2c-2daa-4eeb-a0b8-cbd44355537d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:19.792698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1342/2682] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:51:19.811511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:19.820350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-2dd5ce2c-2daa-4eeb-a0b8-cbd44355537d None None] MFA Rules not processed for user `1beceb4bb1f24552af7bb04c73b97464`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:19.822272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2dd5ce2c-2daa-4eeb-a0b8-cbd44355537d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:19.895426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1341/2683] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:19.900575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc873081-43a3-49b3-a43e-b71e4e0c0534 service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:19.911373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc873081-43a3-49b3-a43e-b71e4e0c0534 service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:19.971263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a61cd1-0749-420e-b033-804201d2b933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:19.971263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a61cd1-0749-420e-b033-804201d2b933 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:19.971263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a61cd1-0749-420e-b033-804201d2b933 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:19.971588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a61cd1-0749-420e-b033-804201d2b933 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:19.972303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a61cd1-0749-420e-b033-804201d2b933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:19.974219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a61cd1-0749-420e-b033-804201d2b933 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:20.048794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1343/2684] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:51:19 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:51:20.052919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5ce2c-2daa-4eeb-a0b8-cbd44355537d None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:20.063889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5ce2c-2daa-4eeb-a0b8-cbd44355537d None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:20.120763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcb867-2650-49cc-96a1-c88642373775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:20.121378 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcb867-2650-49cc-96a1-c88642373775 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:20.121657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcb867-2650-49cc-96a1-c88642373775 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:20.121929 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcb867-2650-49cc-96a1-c88642373775 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:20.122875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcb867-2650-49cc-96a1-c88642373775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:20.124806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcb867-2650-49cc-96a1-c88642373775 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:20.241618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1342/2685] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:51:20 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:51:20.246502 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a61cd1-0749-420e-b033-804201d2b933 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:20.257209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a61cd1-0749-420e-b033-804201d2b933 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:20.261867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd2373-e421-4468-ad16-d0f80ae0fc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:20.262629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd2373-e421-4468-ad16-d0f80ae0fc07 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:20.262881 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd2373-e421-4468-ad16-d0f80ae0fc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:20.263157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd2373-e421-4468-ad16-d0f80ae0fc07 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:20.263838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dd2373-e421-4468-ad16-d0f80ae0fc07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:20.265408 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dd2373-e421-4468-ad16-d0f80ae0fc07 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:20.282768 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1344/2686] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:51:20.286808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcb867-2650-49cc-96a1-c88642373775 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:20.297419 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcb867-2650-49cc-96a1-c88642373775 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:20.302056 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f2f59-672a-4432-9984-e6a4ba4e8fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:20.302710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f2f59-672a-4432-9984-e6a4ba4e8fcd admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:20.302995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f2f59-672a-4432-9984-e6a4ba4e8fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:20.303271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f2f59-672a-4432-9984-e6a4ba4e8fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:20.303965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f2f59-672a-4432-9984-e6a4ba4e8fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:20.305542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f2f59-672a-4432-9984-e6a4ba4e8fcd admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:20.315433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1343/2687] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:51:20.319999 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd2373-e421-4468-ad16-d0f80ae0fc07 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:20.329942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd2373-e421-4468-ad16-d0f80ae0fc07 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:20.334516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2734bf9-7f77-42f9-b303-3aa1169399b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:20.335167 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2734bf9-7f77-42f9-b303-3aa1169399b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:20.339983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2734bf9-7f77-42f9-b303-3aa1169399b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:20.339983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2734bf9-7f77-42f9-b303-3aa1169399b2 admin admin] PATH_INFO: `/v3/projects/8a71f1e0abed4bf2a1bfe4ad80f9ca29/users/b6c9266444704621a3644031f410029e/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:20.351049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2734bf9-7f77-42f9-b303-3aa1169399b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a71f1e0abed4bf2a1bfe4ad80f9ca29, user_id=b6c9266444704621a3644031f410029e, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:20.354531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2734bf9-7f77-42f9-b303-3aa1169399b2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:20.470529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1345/2688] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:51:20 2022] PUT /identity/v3/projects/8a71f1e0abed4bf2a1bfe4ad80f9ca29/users/b6c9266444704621a3644031f410029e/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:51:20.480719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570ad90-3b48-4569-af9f-f8c92b497665 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:20.481278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570ad90-3b48-4569-af9f-f8c92b497665 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:20.481278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570ad90-3b48-4569-af9f-f8c92b497665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:20.512413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-f570ad90-3b48-4569-af9f-f8c92b497665 None None] MFA Rules not processed for user `b6c9266444704621a3644031f410029e`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:20.513646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f570ad90-3b48-4569-af9f-f8c92b497665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:20.588539 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1344/2689] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:51:20.593104 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2734bf9-7f77-42f9-b303-3aa1169399b2 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:20.656311 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2734bf9-7f77-42f9-b303-3aa1169399b2 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:20.661183 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e39a09-911f-45dc-a0d2-a0ad84db4b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:20.661799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e39a09-911f-45dc-a0d2-a0ad84db4b88 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:20.662113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e39a09-911f-45dc-a0d2-a0ad84db4b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:20.662392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e39a09-911f-45dc-a0d2-a0ad84db4b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:20.663086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e39a09-911f-45dc-a0d2-a0ad84db4b88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:20.665954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e39a09-911f-45dc-a0d2-a0ad84db4b88 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:20.750757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:20.757034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1346/2690] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:51:20 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:51:20.761494 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570ad90-3b48-4569-af9f-f8c92b497665 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:20.772222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570ad90-3b48-4569-af9f-f8c92b497665 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:20.776732 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:20.829126 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beee173-7a40-4a0b-b425-0f06abbf498e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:20.829669 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee173-7a40-4a0b-b425-0f06abbf498e admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:20.829951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee173-7a40-4a0b-b425-0f06abbf498e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:20.830256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee173-7a40-4a0b-b425-0f06abbf498e admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:20.830927 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beee173-7a40-4a0b-b425-0f06abbf498e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:20.832835 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beee173-7a40-4a0b-b425-0f06abbf498e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:20.933665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1345/2691] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:51:20 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:51:20.937778 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e39a09-911f-45dc-a0d2-a0ad84db4b88 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:20.948749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e39a09-911f-45dc-a0d2-a0ad84db4b88 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:20.953388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954153ed-a05d-436e-9cc0-c43c25eb0204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:20.954119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954153ed-a05d-436e-9cc0-c43c25eb0204 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:20.954401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954153ed-a05d-436e-9cc0-c43c25eb0204 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:20.954701 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954153ed-a05d-436e-9cc0-c43c25eb0204 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:20.955381 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954153ed-a05d-436e-9cc0-c43c25eb0204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:20.956951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954153ed-a05d-436e-9cc0-c43c25eb0204 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:20.967645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1347/2692] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:51:20.971977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beee173-7a40-4a0b-b425-0f06abbf498e admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:20.982561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beee173-7a40-4a0b-b425-0f06abbf498e admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:20.987202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5a99e-59a3-42b1-aee1-dafd7ae9e2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:20.987852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a99e-59a3-42b1-aee1-dafd7ae9e2da admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:20.988168 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a99e-59a3-42b1-aee1-dafd7ae9e2da admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:20.988445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a99e-59a3-42b1-aee1-dafd7ae9e2da admin admin] PATH_INFO: `/v3/projects/e3107909ab1d4a089504cd51b5cf1e7b/users/0f220b28fa9443aeb210eef334484b05/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:21.003637 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5a99e-59a3-42b1-aee1-dafd7ae9e2da admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3107909ab1d4a089504cd51b5cf1e7b, user_id=0f220b28fa9443aeb210eef334484b05, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:21.006290 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5a99e-59a3-42b1-aee1-dafd7ae9e2da admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:21.070177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1346/2693] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:51:20 2022] PUT /identity/v3/projects/e3107909ab1d4a089504cd51b5cf1e7b/users/0f220b28fa9443aeb210eef334484b05/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:51:21.074382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954153ed-a05d-436e-9cc0-c43c25eb0204 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:21.087485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954153ed-a05d-436e-9cc0-c43c25eb0204 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:21.143865 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289a549-9db6-46d1-8750-7765f0624b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:21.144484 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289a549-9db6-46d1-8750-7765f0624b80 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:21.144770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289a549-9db6-46d1-8750-7765f0624b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:21.145049 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289a549-9db6-46d1-8750-7765f0624b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:21.145734 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289a549-9db6-46d1-8750-7765f0624b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:21.147409 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289a549-9db6-46d1-8750-7765f0624b80 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:21.158540 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1348/2694] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:51:21.162744 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5a99e-59a3-42b1-aee1-dafd7ae9e2da admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:21.173719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5a99e-59a3-42b1-aee1-dafd7ae9e2da admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:21.178472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7527d2-bef4-4db9-a4b3-4e03611fd0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:21.179146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7527d2-bef4-4db9-a4b3-4e03611fd0a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:21.179421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7527d2-bef4-4db9-a4b3-4e03611fd0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:21.179695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7527d2-bef4-4db9-a4b3-4e03611fd0a6 admin admin] PATH_INFO: `/v3/projects/e3107909ab1d4a089504cd51b5cf1e7b/users/0f220b28fa9443aeb210eef334484b05/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:21.181905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7527d2-bef4-4db9-a4b3-4e03611fd0a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3107909ab1d4a089504cd51b5cf1e7b, user_id=0f220b28fa9443aeb210eef334484b05, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:21.184478 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7527d2-bef4-4db9-a4b3-4e03611fd0a6 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:21.195095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1347/2695] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:51:21 2022] PUT /identity/v3/projects/e3107909ab1d4a089504cd51b5cf1e7b/users/0f220b28fa9443aeb210eef334484b05/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:51:21.206823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088586-8160-4f55-8988-af256d03ad1c None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:21.207118 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088586-8160-4f55-8988-af256d03ad1c None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:21.207440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088586-8160-4f55-8988-af256d03ad1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:21.237033 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-85088586-8160-4f55-8988-af256d03ad1c None None] MFA Rules not processed for user `0f220b28fa9443aeb210eef334484b05`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:21.238297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-85088586-8160-4f55-8988-af256d03ad1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:21.309269 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1349/2696] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:21.320765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7527d2-bef4-4db9-a4b3-4e03611fd0a6 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:21.377020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7527d2-bef4-4db9-a4b3-4e03611fd0a6 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:21.380612 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba901e89-eb00-436d-8c2a-03c831ddfa91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:21.381251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba901e89-eb00-436d-8c2a-03c831ddfa91 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:21.381528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba901e89-eb00-436d-8c2a-03c831ddfa91 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:21.381816 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba901e89-eb00-436d-8c2a-03c831ddfa91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:21.437882 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba901e89-eb00-436d-8c2a-03c831ddfa91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:21.439344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba901e89-eb00-436d-8c2a-03c831ddfa91 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:21.448697 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1348/2697] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:51:24.479664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85088586-8160-4f55-8988-af256d03ad1c None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:24.489704 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85088586-8160-4f55-8988-af256d03ad1c None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:24.493435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e927e-decd-41e2-8bbe-3289da96c36d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:24.494075 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e927e-decd-41e2-8bbe-3289da96c36d service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:24.494385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e927e-decd-41e2-8bbe-3289da96c36d service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:24.494659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e927e-decd-41e2-8bbe-3289da96c36d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:24.565042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e927e-decd-41e2-8bbe-3289da96c36d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:24.566496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e927e-decd-41e2-8bbe-3289da96c36d service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:24.576034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1350/2698] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:51:26.051308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba901e89-eb00-436d-8c2a-03c831ddfa91 service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:26.060083 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba901e89-eb00-436d-8c2a-03c831ddfa91 service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:26.063999 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc45ea4-8b02-4e4b-95ba-a3c607313d43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:26.065525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc45ea4-8b02-4e4b-95ba-a3c607313d43 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:26.065525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc45ea4-8b02-4e4b-95ba-a3c607313d43 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:26.065525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc45ea4-8b02-4e4b-95ba-a3c607313d43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:26.075092 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc45ea4-8b02-4e4b-95ba-a3c607313d43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:26.076497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc45ea4-8b02-4e4b-95ba-a3c607313d43 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:26.081589 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:26.086461 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1349/2699] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:51:28.761050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e927e-decd-41e2-8bbe-3289da96c36d service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:28.762634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6e7e927e-decd-41e2-8bbe-3289da96c36d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:28.773866 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=100 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:28.820021 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e927e-decd-41e2-8bbe-3289da96c36d service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:28.824180 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481db09-f430-49e2-a097-635b372d9491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:28.825003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481db09-f430-49e2-a097-635b372d9491 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:28.825281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481db09-f430-49e2-a097-635b372d9491 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:28.825663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481db09-f430-49e2-a097-635b372d9491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:28.827440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2481db09-f430-49e2-a097-635b372d9491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:28.891158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481db09-f430-49e2-a097-635b372d9491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:28.892597 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481db09-f430-49e2-a097-635b372d9491 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:28.903976 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1351/2700] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:51:31.757855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc45ea4-8b02-4e4b-95ba-a3c607313d43 service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:31.764217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc45ea4-8b02-4e4b-95ba-a3c607313d43 service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:31.768080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fa5c1-abdc-4826-9b7c-fa65901d81c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:31.768747 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fa5c1-abdc-4826-9b7c-fa65901d81c9 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:31.769199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fa5c1-abdc-4826-9b7c-fa65901d81c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:31.769491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fa5c1-abdc-4826-9b7c-fa65901d81c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:31.771324 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-417fa5c1-abdc-4826-9b7c-fa65901d81c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:31.781594 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fa5c1-abdc-4826-9b7c-fa65901d81c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:31.783118 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fa5c1-abdc-4826-9b7c-fa65901d81c9 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:31.795450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1350/2701] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:51:31.874048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481db09-f430-49e2-a097-635b372d9491 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:31.884878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481db09-f430-49e2-a097-635b372d9491 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:31.890861 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:31.891489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:31.891774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:31.892051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] PATH_INFO: `/v3/users/1beceb4bb1f24552af7bb04c73b97464` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:31.893278 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1beceb4bb1f24552af7bb04c73b97464)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:31.895261 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:32.110269 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1beceb4bb1f24552af7bb04c73b97464'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:32.229669 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1beceb4bb1f24552af7bb04c73b97464'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:32.244264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1beceb4bb1f24552af7bb04c73b97464'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:32.244863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1beceb4bb1f24552af7bb04c73b97464'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:32.252962 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1beceb4bb1f24552af7bb04c73b97464'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:32.259175 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1352/2702] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:51:31 2022] DELETE /identity/v3/users/1beceb4bb1f24552af7bb04c73b97464 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:51:32.263930 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb68ef-dcd4-4a53-9772-2cb4ef9c0bc8 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:32.264213 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb68ef-dcd4-4a53-9772-2cb4ef9c0bc8 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:32.264488 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb68ef-dcd4-4a53-9772-2cb4ef9c0bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:32.284477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-46cb68ef-dcd4-4a53-9772-2cb4ef9c0bc8 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:32.291250 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-46cb68ef-dcd4-4a53-9772-2cb4ef9c0bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:32.359618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1351/2703] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:32.400531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:32.402849 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:32.444756 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:32.473776 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2af77-1ec3-4dc1-809b-3e8326e7e75d admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:32.478106 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c26e20-2636-429e-828b-d5d81cfa481e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:32.478892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c26e20-2636-429e-828b-d5d81cfa481e service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:32.479158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c26e20-2636-429e-828b-d5d81cfa481e service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:32.479478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c26e20-2636-429e-828b-d5d81cfa481e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:32.491001 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c26e20-2636-429e-828b-d5d81cfa481e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:32.492945 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c26e20-2636-429e-828b-d5d81cfa481e service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:32.505927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1353/2704] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:51:33.111464 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb68ef-dcd4-4a53-9772-2cb4ef9c0bc8 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:33.112985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-46cb68ef-dcd4-4a53-9772-2cb4ef9c0bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:33.126333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb68ef-dcd4-4a53-9772-2cb4ef9c0bc8 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:33.130992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:33.131671 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:33.131943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:33.132276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] PATH_INFO: `/v3/projects/4bc3f55f81ae45a8906d2349f694ef53` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:33.133435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc3f55f81ae45a8906d2349f694ef53)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:33.135379 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:33.254459 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c26e20-2636-429e-828b-d5d81cfa481e service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:33.255760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-23c26e20-2636-429e-828b-d5d81cfa481e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:33.270405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c26e20-2636-429e-828b-d5d81cfa481e service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:33.275078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:33.275713 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:33.276003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:33.276275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] PATH_INFO: `/v3/users/95fb2edf82644e7f8ce461fcbb52007b` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:33.278566 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95fb2edf82644e7f8ce461fcbb52007b)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:33.282550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:33.294921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc3f55f81ae45a8906d2349f694ef53', 'request_id': 'req-14e05c51-fcca-4d4e-8741-e1db75cb5276'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.318985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc3f55f81ae45a8906d2349f694ef53', 'request_id': 'req-14e05c51-fcca-4d4e-8741-e1db75cb5276'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.436014 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc3f55f81ae45a8906d2349f694ef53', 'request_id': 'req-14e05c51-fcca-4d4e-8741-e1db75cb5276'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.447228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] The token cache is being invalidate because project 4bc3f55f81ae45a8906d2349f694ef53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:51:33.447595 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.448180 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.449708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1352/2705] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:51:33 2022] DELETE /identity/v3/projects/4bc3f55f81ae45a8906d2349f694ef53 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:51:33.453578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:33.454970 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:33.521418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95fb2edf82644e7f8ce461fcbb52007b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.525985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e05c51-fcca-4d4e-8741-e1db75cb5276 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:33.533066 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:33.533772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:33.534091 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:33.534396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] PATH_INFO: `/v3/users/b6c9266444704621a3644031f410029e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:33.535618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c9266444704621a3644031f410029e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:33.544132 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:33.558706 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:33.664061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95fb2edf82644e7f8ce461fcbb52007b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.679858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95fb2edf82644e7f8ce461fcbb52007b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.685829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95fb2edf82644e7f8ce461fcbb52007b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.690140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95fb2edf82644e7f8ce461fcbb52007b'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.699219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1354/2706] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:51:33 2022] DELETE /identity/v3/users/95fb2edf82644e7f8ce461fcbb52007b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:51:33.805128 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c9266444704621a3644031f410029e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.938111 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c9266444704621a3644031f410029e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.938921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c9266444704621a3644031f410029e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.955172 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c9266444704621a3644031f410029e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.964868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c9266444704621a3644031f410029e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:33.974588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1353/2707] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:51:33 2022] DELETE /identity/v3/users/b6c9266444704621a3644031f410029e => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:51:34.000216 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:34.001657 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:34.081438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5418d0-71ab-46aa-a10f-5b5fac0386a3 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:34.131273 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:34.141101 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:34.141764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:34.142130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:34.142405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] PATH_INFO: `/v3/projects/4b95ce24a4d34e379fa86c9114ece400` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:34.143628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b95ce24a4d34e379fa86c9114ece400)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:34.145517 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:34.267252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b95ce24a4d34e379fa86c9114ece400', 'request_id': 'req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:34.389405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b95ce24a4d34e379fa86c9114ece400', 'request_id': 'req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:34.395798 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b95ce24a4d34e379fa86c9114ece400', 'request_id': 'req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:34.407821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] The token cache is being invalidate because project 4b95ce24a4d34e379fa86c9114ece400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:51:34.408115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:34.408924 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-c2e8a0d8-c5f1-42c6-9440-0d47475e29d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:34.410496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1355/2708] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:51:33 2022] DELETE /identity/v3/projects/4b95ce24a4d34e379fa86c9114ece400 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:51:34.414146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:34.416538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:34.419720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d87ae-b425-4fd8-9964-e2866371383a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:34.420050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d87ae-b425-4fd8-9964-e2866371383a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:34.420406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d87ae-b425-4fd8-9964-e2866371383a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:34.431012 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13296c67-1166-4591-9f52-a23e8c37500f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:34.458084 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-d82d87ae-b425-4fd8-9964-e2866371383a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:34.459632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d82d87ae-b425-4fd8-9964-e2866371383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:34.504764 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:34.505775 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:34.506102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:34.506448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] PATH_INFO: `/v3/projects/8a71f1e0abed4bf2a1bfe4ad80f9ca29` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:34.508996 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a71f1e0abed4bf2a1bfe4ad80f9ca29)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:34.513385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:34.545341 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1356/2709] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:34.553086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac649c-f9a3-4e92-aeb4-0af8bad2b2d6 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:34.554653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac649c-f9a3-4e92-aeb4-0af8bad2b2d6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:34.554653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac649c-f9a3-4e92-aeb4-0af8bad2b2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:34.575935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1eac649c-f9a3-4e92-aeb4-0af8bad2b2d6 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:34.577144 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1eac649c-f9a3-4e92-aeb4-0af8bad2b2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:34.585773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1357/2710] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:51:34.590109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac649c-f9a3-4e92-aeb4-0af8bad2b2d6 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:34.599661 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac649c-f9a3-4e92-aeb4-0af8bad2b2d6 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:34.604310 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc5659-b272-43f1-a555-c0caea08c685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:34.608687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5659-b272-43f1-a555-c0caea08c685 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:34.608687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5659-b272-43f1-a555-c0caea08c685 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:34.608687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5659-b272-43f1-a555-c0caea08c685 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:34.608687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc5659-b272-43f1-a555-c0caea08c685 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:51:34.608687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc5659-b272-43f1-a555-c0caea08c685 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:34.608687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc5659-b272-43f1-a555-c0caea08c685 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:34.619550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1358/2711] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:51:34.624595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc5659-b272-43f1-a555-c0caea08c685 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:34.633855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc5659-b272-43f1-a555-c0caea08c685 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:34.639146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851da8a-51c4-438f-bc5c-d1a996cbe0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:34.639718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851da8a-51c4-438f-bc5c-d1a996cbe0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:34.640053 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851da8a-51c4-438f-bc5c-d1a996cbe0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:34.640279 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851da8a-51c4-438f-bc5c-d1a996cbe0d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:34.640932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851da8a-51c4-438f-bc5c-d1a996cbe0d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:34.642816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851da8a-51c4-438f-bc5c-d1a996cbe0d7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:34.664524 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a71f1e0abed4bf2a1bfe4ad80f9ca29', 'request_id': 'req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:34.670909 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a71f1e0abed4bf2a1bfe4ad80f9ca29', 'request_id': 'req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:34.808709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1359/2712] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:51:34 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:51:34.812614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851da8a-51c4-438f-bc5c-d1a996cbe0d7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:34.817746 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a71f1e0abed4bf2a1bfe4ad80f9ca29', 'request_id': 'req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:34.830605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] The token cache is being invalidate because project 8a71f1e0abed4bf2a1bfe4ad80f9ca29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:51:34.831581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:34.832275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:34.833792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1354/2713] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:51:34 2022] DELETE /identity/v3/projects/8a71f1e0abed4bf2a1bfe4ad80f9ca29 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:51:34.837858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:34.839388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:34.860768 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:34.861995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:34.880006 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851da8a-51c4-438f-bc5c-d1a996cbe0d7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:34.884869 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d232eea-2c27-430c-b3fe-4b6bee1a96d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:34.885518 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d232eea-2c27-430c-b3fe-4b6bee1a96d4 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:34.885800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d232eea-2c27-430c-b3fe-4b6bee1a96d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:34.889286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d232eea-2c27-430c-b3fe-4b6bee1a96d4 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:34.889286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d232eea-2c27-430c-b3fe-4b6bee1a96d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:34.889286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d232eea-2c27-430c-b3fe-4b6bee1a96d4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:34.904919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc71a75-bc3b-4d34-8d58-df198ee7f5b9 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:34.909640 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:34.910304 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:34.910586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:34.910855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] PATH_INFO: `/v3/users/0f220b28fa9443aeb210eef334484b05` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:34.912009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f220b28fa9443aeb210eef334484b05)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:34.913903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:35.006851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1360/2714] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:51:34 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:51:35.010914 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d232eea-2c27-430c-b3fe-4b6bee1a96d4 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:35.012595 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6d232eea-2c27-430c-b3fe-4b6bee1a96d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:35.024736 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d232eea-2c27-430c-b3fe-4b6bee1a96d4 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:35.029594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657855fd-fd30-4dad-a99a-cec91a233c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:35.030216 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657855fd-fd30-4dad-a99a-cec91a233c36 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:35.030493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657855fd-fd30-4dad-a99a-cec91a233c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:35.030760 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657855fd-fd30-4dad-a99a-cec91a233c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:35.031437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657855fd-fd30-4dad-a99a-cec91a233c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:35.032993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657855fd-fd30-4dad-a99a-cec91a233c36 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:35.042569 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1361/2715] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:51:35.047356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657855fd-fd30-4dad-a99a-cec91a233c36 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:35.056465 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657855fd-fd30-4dad-a99a-cec91a233c36 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:35.068443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af08e-30f8-4c87-a44e-37d785e3b282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:35.068443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af08e-30f8-4c87-a44e-37d785e3b282 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:35.068443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af08e-30f8-4c87-a44e-37d785e3b282 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:35.068443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af08e-30f8-4c87-a44e-37d785e3b282 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:35.068935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16af08e-30f8-4c87-a44e-37d785e3b282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:35.068935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16af08e-30f8-4c87-a44e-37d785e3b282 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:35.075676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1362/2716] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:51:35.079746 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af08e-30f8-4c87-a44e-37d785e3b282 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:35.088975 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af08e-30f8-4c87-a44e-37d785e3b282 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:35.093618 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5b3b4-427f-4eb3-8c5f-95f99c533338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:35.094286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b3b4-427f-4eb3-8c5f-95f99c533338 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:35.094569 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b3b4-427f-4eb3-8c5f-95f99c533338 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:35.094841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b3b4-427f-4eb3-8c5f-95f99c533338 admin admin] PATH_INFO: `/v3/projects/8c4ab3c7309249228e42c226277fb14f/users/a022ce50f3334e919dcf0008bb6afcf8/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:35.110234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5b3b4-427f-4eb3-8c5f-95f99c533338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c4ab3c7309249228e42c226277fb14f, user_id=a022ce50f3334e919dcf0008bb6afcf8, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:35.112942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5b3b4-427f-4eb3-8c5f-95f99c533338 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:35.229423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1363/2717] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:51:35 2022] PUT /identity/v3/projects/8c4ab3c7309249228e42c226277fb14f/users/a022ce50f3334e919dcf0008bb6afcf8/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:51:35.235316 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912dede-2bcb-4709-9bd0-1d23ad64a7ae None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:35.235607 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912dede-2bcb-4709-9bd0-1d23ad64a7ae None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:35.235888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912dede-2bcb-4709-9bd0-1d23ad64a7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:35.238334 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f220b28fa9443aeb210eef334484b05'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:35.255250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-8912dede-2bcb-4709-9bd0-1d23ad64a7ae None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:35.256472 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8912dede-2bcb-4709-9bd0-1d23ad64a7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:35.318018 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1364/2718] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:51:35.328829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912dede-2bcb-4709-9bd0-1d23ad64a7ae None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:35.330218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8912dede-2bcb-4709-9bd0-1d23ad64a7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:35.355301 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f220b28fa9443aeb210eef334484b05'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:35.356445 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f220b28fa9443aeb210eef334484b05'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:35.369818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f220b28fa9443aeb210eef334484b05'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:35.380737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f220b28fa9443aeb210eef334484b05'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:35.387242 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1355/2719] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:51:34 2022] DELETE /identity/v3/users/0f220b28fa9443aeb210eef334484b05 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:51:35.397165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912dede-2bcb-4709-9bd0-1d23ad64a7ae None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:35.442209 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:35.452297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15617e8b-cdba-45e4-8bf6-6bb686e4656b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:35.453009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15617e8b-cdba-45e4-8bf6-6bb686e4656b service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:35.453327 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15617e8b-cdba-45e4-8bf6-6bb686e4656b service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:35.453622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15617e8b-cdba-45e4-8bf6-6bb686e4656b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:35.455510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-15617e8b-cdba-45e4-8bf6-6bb686e4656b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:35.526408 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15617e8b-cdba-45e4-8bf6-6bb686e4656b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:35.529971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15617e8b-cdba-45e4-8bf6-6bb686e4656b service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:35.538749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1365/2720] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:51:35.920275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:35.921918 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:35.944791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bcaa1-f012-4be1-b1c1-873e821c95c5 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:35.950276 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:35.950981 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:35.951285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:35.951484 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] PATH_INFO: `/v3/projects/e3107909ab1d4a089504cd51b5cf1e7b` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:35.952831 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3107909ab1d4a089504cd51b5cf1e7b)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:35.954773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:36.071285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3107909ab1d4a089504cd51b5cf1e7b', 'request_id': 'req-ab4786a6-a778-49a8-a2a9-03bfc8722188'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:36.077826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3107909ab1d4a089504cd51b5cf1e7b', 'request_id': 'req-ab4786a6-a778-49a8-a2a9-03bfc8722188'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:36.161259 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3107909ab1d4a089504cd51b5cf1e7b', 'request_id': 'req-ab4786a6-a778-49a8-a2a9-03bfc8722188'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:36.176660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] The token cache is being invalidate because project e3107909ab1d4a089504cd51b5cf1e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:51:36.177029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:36.177716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-ab4786a6-a778-49a8-a2a9-03bfc8722188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:51:36.180017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1356/2721] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:51:35 2022] DELETE /identity/v3/projects/e3107909ab1d4a089504cd51b5cf1e7b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:51:36.188024 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3107c-70f7-4ab9-b07d-a4a518bc7394 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:36.188404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3107c-70f7-4ab9-b07d-a4a518bc7394 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:36.189828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3107c-70f7-4ab9-b07d-a4a518bc7394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:36.212878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1ad3107c-70f7-4ab9-b07d-a4a518bc7394 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:36.214196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1ad3107c-70f7-4ab9-b07d-a4a518bc7394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:36.284726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1366/2722] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:36.292495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf5511-622f-4dc9-8f53-3b7af649f8d6 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:36.292785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf5511-622f-4dc9-8f53-3b7af649f8d6 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:36.293060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf5511-622f-4dc9-8f53-3b7af649f8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:36.312891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-6ecf5511-622f-4dc9-8f53-3b7af649f8d6 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:36.314137 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6ecf5511-622f-4dc9-8f53-3b7af649f8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:36.322557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1357/2723] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:36.327541 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3107c-70f7-4ab9-b07d-a4a518bc7394 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:36.339936 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3107c-70f7-4ab9-b07d-a4a518bc7394 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:36.346506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6259f18-e8b0-4450-bb49-e03d824782c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:36.347136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6259f18-e8b0-4450-bb49-e03d824782c8 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:36.347419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6259f18-e8b0-4450-bb49-e03d824782c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:36.347695 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6259f18-e8b0-4450-bb49-e03d824782c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:36.348401 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6259f18-e8b0-4450-bb49-e03d824782c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:51:36.348710 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6259f18-e8b0-4450-bb49-e03d824782c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:36.350339 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6259f18-e8b0-4450-bb49-e03d824782c8 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:36.364075 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1367/2724] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:51:36.367948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf5511-622f-4dc9-8f53-3b7af649f8d6 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:36.378523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf5511-622f-4dc9-8f53-3b7af649f8d6 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:36.383491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372fa1d-62b6-44b6-8ef2-01e10b97840a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:36.390584 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372fa1d-62b6-44b6-8ef2-01e10b97840a admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:36.390584 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372fa1d-62b6-44b6-8ef2-01e10b97840a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:36.390584 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372fa1d-62b6-44b6-8ef2-01e10b97840a admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:36.390584 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372fa1d-62b6-44b6-8ef2-01e10b97840a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:36.390584 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372fa1d-62b6-44b6-8ef2-01e10b97840a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:36.404819 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:36.574429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1358/2725] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:51:36 2022] POST /identity/v3/projects => generated 391 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:51:36.579633 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6259f18-e8b0-4450-bb49-e03d824782c8 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:36.589498 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6259f18-e8b0-4450-bb49-e03d824782c8 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:36.648564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c2c79-2f23-4301-9c01-f3912c1d323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:36.649184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c2c79-2f23-4301-9c01-f3912c1d323e admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:36.649468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c2c79-2f23-4301-9c01-f3912c1d323e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:36.649739 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c2c79-2f23-4301-9c01-f3912c1d323e admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:36.650444 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c2c79-2f23-4301-9c01-f3912c1d323e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:36.652428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c2c79-2f23-4301-9c01-f3912c1d323e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:36.791598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1368/2726] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:51:36 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:51:36.795348 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372fa1d-62b6-44b6-8ef2-01e10b97840a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:36.807095 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372fa1d-62b6-44b6-8ef2-01e10b97840a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:36.813519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d8b32-a767-4041-8174-6325afb50318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:36.813519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8b32-a767-4041-8174-6325afb50318 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:36.813519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8b32-a767-4041-8174-6325afb50318 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:36.813519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8b32-a767-4041-8174-6325afb50318 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:36.814284 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d8b32-a767-4041-8174-6325afb50318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:36.816024 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d8b32-a767-4041-8174-6325afb50318 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:36.828197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1359/2727] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:51:36.832262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c2c79-2f23-4301-9c01-f3912c1d323e admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:36.842743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c2c79-2f23-4301-9c01-f3912c1d323e admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:36.847325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdd608-f930-4821-88dc-9bfcc9d65425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:36.847923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdd608-f930-4821-88dc-9bfcc9d65425 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:36.848204 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdd608-f930-4821-88dc-9bfcc9d65425 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:36.849133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdd608-f930-4821-88dc-9bfcc9d65425 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:36.849133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdd608-f930-4821-88dc-9bfcc9d65425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:36.850800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdd608-f930-4821-88dc-9bfcc9d65425 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:36.852918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:36.861829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1369/2728] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:51:36.866227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d8b32-a767-4041-8174-6325afb50318 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:36.877194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d8b32-a767-4041-8174-6325afb50318 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:36.881752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c21bf-5cbf-4660-a963-e8d09b9af116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:36.882435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c21bf-5cbf-4660-a963-e8d09b9af116 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:36.882714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c21bf-5cbf-4660-a963-e8d09b9af116 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:36.882984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c21bf-5cbf-4660-a963-e8d09b9af116 admin admin] PATH_INFO: `/v3/projects/9292e8e5d43e4dfbb2be737a56b4e562/users/dfe7d2c321fc4d4588a6afb8801a94c7/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:36.898538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c21bf-5cbf-4660-a963-e8d09b9af116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9292e8e5d43e4dfbb2be737a56b4e562, user_id=dfe7d2c321fc4d4588a6afb8801a94c7, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:36.901190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c21bf-5cbf-4660-a963-e8d09b9af116 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:37.008209 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1360/2729] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:51:36 2022] PUT /identity/v3/projects/9292e8e5d43e4dfbb2be737a56b4e562/users/dfe7d2c321fc4d4588a6afb8801a94c7/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:51:37.023853 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bec00-b4c7-4a77-890c-9510fb71a787 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:37.024215 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bec00-b4c7-4a77-890c-9510fb71a787 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:37.024412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bec00-b4c7-4a77-890c-9510fb71a787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:37.053372 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-942bec00-b4c7-4a77-890c-9510fb71a787 None None] MFA Rules not processed for user `dfe7d2c321fc4d4588a6afb8801a94c7`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:37.054641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-942bec00-b4c7-4a77-890c-9510fb71a787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:37.123978 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1370/2730] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:37.134719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c21bf-5cbf-4660-a963-e8d09b9af116 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:37.136609 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1b5c21bf-5cbf-4660-a963-e8d09b9af116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:37.198204 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c21bf-5cbf-4660-a963-e8d09b9af116 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:37.202766 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498e0a6-e2ad-42ec-9f68-f74f46859e95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:37.203396 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e0a6-e2ad-42ec-9f68-f74f46859e95 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:37.207901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e0a6-e2ad-42ec-9f68-f74f46859e95 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:37.207901 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e0a6-e2ad-42ec-9f68-f74f46859e95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:37.212579 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498e0a6-e2ad-42ec-9f68-f74f46859e95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:37.214037 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498e0a6-e2ad-42ec-9f68-f74f46859e95 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:37.223432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1361/2731] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:51:41.476490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b77384-bcbc-4aed-8bd6-9ce867c4a160 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:41.476996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b77384-bcbc-4aed-8bd6-9ce867c4a160 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:41.477113 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b77384-bcbc-4aed-8bd6-9ce867c4a160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:41.516610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-b1b77384-bcbc-4aed-8bd6-9ce867c4a160 None None] MFA Rules not processed for user `a022ce50f3334e919dcf0008bb6afcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:41.516610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b1b77384-bcbc-4aed-8bd6-9ce867c4a160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:41.603845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1371/2732] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:41.629106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498e0a6-e2ad-42ec-9f68-f74f46859e95 service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:41.630607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c498e0a6-e2ad-42ec-9f68-f74f46859e95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:41.731542 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498e0a6-e2ad-42ec-9f68-f74f46859e95 service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:41.737227 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee4167-4096-457f-aa7e-8e80a444cbc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:41.737838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee4167-4096-457f-aa7e-8e80a444cbc2 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:41.738155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee4167-4096-457f-aa7e-8e80a444cbc2 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:41.738432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee4167-4096-457f-aa7e-8e80a444cbc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:41.749814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee4167-4096-457f-aa7e-8e80a444cbc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:41.752835 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee4167-4096-457f-aa7e-8e80a444cbc2 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:41.764950 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1362/2733] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:51:44.434963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56031b-362e-4e99-9c10-18a9c662f061 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:44.435550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56031b-362e-4e99-9c10-18a9c662f061 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:44.435550 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56031b-362e-4e99-9c10-18a9c662f061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:44.454826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-db56031b-362e-4e99-9c10-18a9c662f061 None None] MFA Rules not processed for user `a022ce50f3334e919dcf0008bb6afcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:51:44.456088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-db56031b-362e-4e99-9c10-18a9c662f061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:44.469720 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1372/2734] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:51:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:51:51.759472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:INSERT=2,keystone:SELECT=77 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:51:53.146071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee4167-4096-457f-aa7e-8e80a444cbc2 service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:51:53.160135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee4167-4096-457f-aa7e-8e80a444cbc2 service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:51:53.170217 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a61521-64ab-45a7-9427-8e3c8c2ad232 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:51:53.175714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a61521-64ab-45a7-9427-8e3c8c2ad232 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:51:53.176027 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a61521-64ab-45a7-9427-8e3c8c2ad232 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:51:53.176278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a61521-64ab-45a7-9427-8e3c8c2ad232 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:51:53.182375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f7a61521-64ab-45a7-9427-8e3c8c2ad232 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:51:53.197506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a61521-64ab-45a7-9427-8e3c8c2ad232 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:51:53.199121 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a61521-64ab-45a7-9427-8e3c8c2ad232 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:51:53.211405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1363/2735] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:51:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:51:54.449896 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=68 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:52:03.207057 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=6 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:52:03.862961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56031b-362e-4e99-9c10-18a9c662f061 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:03.925220 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56031b-362e-4e99-9c10-18a9c662f061 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:03.929670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:03.930419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:03.930722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:03.930992 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] PATH_INFO: `/v3/users/dfe7d2c321fc4d4588a6afb8801a94c7` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:03.932146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfe7d2c321fc4d4588a6afb8801a94c7)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:03.935816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:04.192642 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfe7d2c321fc4d4588a6afb8801a94c7'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:52:04.308739 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfe7d2c321fc4d4588a6afb8801a94c7'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:52:04.322351 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfe7d2c321fc4d4588a6afb8801a94c7'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:52:04.323127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfe7d2c321fc4d4588a6afb8801a94c7'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:52:04.330438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfe7d2c321fc4d4588a6afb8801a94c7'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:52:04.336299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1373/2736] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:52:03 2022] DELETE /identity/v3/users/dfe7d2c321fc4d4588a6afb8801a94c7 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:52:04.340601 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06b1fc-935d-45cf-96c2-15cf34c04a21 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:04.341061 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06b1fc-935d-45cf-96c2-15cf34c04a21 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:04.341159 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06b1fc-935d-45cf-96c2-15cf34c04a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:04.360826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-de06b1fc-935d-45cf-96c2-15cf34c04a21 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:52:04.362437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-de06b1fc-935d-45cf-96c2-15cf34c04a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:04.421491 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1364/2737] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:52:04.431824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:04.433287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:04.490390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558f41b-0a5e-4ca3-901b-994894dadfb7 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:04.493903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ac83a-e223-4a93-8a8b-1290e068192b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:04.494591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ac83a-e223-4a93-8a8b-1290e068192b service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:04.494871 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ac83a-e223-4a93-8a8b-1290e068192b service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:04.495140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ac83a-e223-4a93-8a8b-1290e068192b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:04.504570 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ac83a-e223-4a93-8a8b-1290e068192b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:04.506094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ac83a-e223-4a93-8a8b-1290e068192b service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:04.515460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1374/2738] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:52:04.984934 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06b1fc-935d-45cf-96c2-15cf34c04a21 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:04.986553 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-de06b1fc-935d-45cf-96c2-15cf34c04a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:04.998910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06b1fc-935d-45cf-96c2-15cf34c04a21 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:05.004387 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:05.005090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:05.005375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:05.005646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] PATH_INFO: `/v3/projects/9292e8e5d43e4dfbb2be737a56b4e562` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:05.006852 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9292e8e5d43e4dfbb2be737a56b4e562)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:05.008759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:05.103586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9292e8e5d43e4dfbb2be737a56b4e562', 'request_id': 'req-01c35fed-de73-43bf-ab3f-2c8bd97936f8'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:52:05.109865 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9292e8e5d43e4dfbb2be737a56b4e562', 'request_id': 'req-01c35fed-de73-43bf-ab3f-2c8bd97936f8'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:52:05.177355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9292e8e5d43e4dfbb2be737a56b4e562', 'request_id': 'req-01c35fed-de73-43bf-ab3f-2c8bd97936f8'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:52:05.189719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] The token cache is being invalidate because project 9292e8e5d43e4dfbb2be737a56b4e562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:52:05.190030 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:52:05.190759 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-01c35fed-de73-43bf-ab3f-2c8bd97936f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:52:05.192295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1365/2739] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:52:04 2022] DELETE /identity/v3/projects/9292e8e5d43e4dfbb2be737a56b4e562 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:52:05.197453 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c748726-02f1-40d4-bf5d-47051a442e9f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:05.197915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c748726-02f1-40d4-bf5d-47051a442e9f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:05.198057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c748726-02f1-40d4-bf5d-47051a442e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:05.218083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-3c748726-02f1-40d4-bf5d-47051a442e9f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:52:05.219439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-3c748726-02f1-40d4-bf5d-47051a442e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:05.237281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:52:05.282146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1375/2740] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:52:05.290255 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec22f3e-df96-439e-884c-95af48853155 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:05.290541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec22f3e-df96-439e-884c-95af48853155 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:05.290815 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec22f3e-df96-439e-884c-95af48853155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:05.310772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-aec22f3e-df96-439e-884c-95af48853155 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:52:05.311955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-aec22f3e-df96-439e-884c-95af48853155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:05.319896 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1366/2741] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:52:05.324202 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c748726-02f1-40d4-bf5d-47051a442e9f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:05.334391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c748726-02f1-40d4-bf5d-47051a442e9f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:05.339031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d17f9-fcc1-48a9-9b86-b5fa9f9dc9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:05.339581 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d17f9-fcc1-48a9-9b86-b5fa9f9dc9dd admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:05.339862 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d17f9-fcc1-48a9-9b86-b5fa9f9dc9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:05.340134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d17f9-fcc1-48a9-9b86-b5fa9f9dc9dd admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:05.340846 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d17f9-fcc1-48a9-9b86-b5fa9f9dc9dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:52:05.341150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d17f9-fcc1-48a9-9b86-b5fa9f9dc9dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:05.342792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d17f9-fcc1-48a9-9b86-b5fa9f9dc9dd admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:05.358832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1376/2742] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:52:05.360619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec22f3e-df96-439e-884c-95af48853155 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:05.370033 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec22f3e-df96-439e-884c-95af48853155 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:05.375252 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419254da-ae17-4686-874b-26a151b297cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:05.376105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419254da-ae17-4686-874b-26a151b297cf admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:05.376487 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419254da-ae17-4686-874b-26a151b297cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:05.376762 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419254da-ae17-4686-874b-26a151b297cf admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:05.377446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419254da-ae17-4686-874b-26a151b297cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:05.379369 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419254da-ae17-4686-874b-26a151b297cf admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:05.491903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1367/2743] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:52:05 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:52:05.493205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d17f9-fcc1-48a9-9b86-b5fa9f9dc9dd admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:05.505951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d17f9-fcc1-48a9-9b86-b5fa9f9dc9dd admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:05.568069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9e606-fc65-4e2e-a12e-996be2dec98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:05.568682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9e606-fc65-4e2e-a12e-996be2dec98b admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:05.568961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9e606-fc65-4e2e-a12e-996be2dec98b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:05.569231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9e606-fc65-4e2e-a12e-996be2dec98b admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:05.569901 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9e606-fc65-4e2e-a12e-996be2dec98b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:05.571834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9e606-fc65-4e2e-a12e-996be2dec98b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:05.674680 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1377/2744] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:52:05 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:52:05.678924 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419254da-ae17-4686-874b-26a151b297cf admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:05.689503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419254da-ae17-4686-874b-26a151b297cf admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:05.694320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eb53a-dcc0-4ab3-9f68-54d672b77f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:05.694860 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eb53a-dcc0-4ab3-9f68-54d672b77f08 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:05.695148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eb53a-dcc0-4ab3-9f68-54d672b77f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:05.695454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eb53a-dcc0-4ab3-9f68-54d672b77f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:05.696280 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3eb53a-dcc0-4ab3-9f68-54d672b77f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:05.697875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3eb53a-dcc0-4ab3-9f68-54d672b77f08 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:05.708412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1368/2745] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:52:05.712443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9e606-fc65-4e2e-a12e-996be2dec98b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:05.723402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9e606-fc65-4e2e-a12e-996be2dec98b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:05.728087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a162a-2623-4831-9b73-e8f33fcdcf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:05.728682 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a162a-2623-4831-9b73-e8f33fcdcf45 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:05.728961 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a162a-2623-4831-9b73-e8f33fcdcf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:05.729232 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a162a-2623-4831-9b73-e8f33fcdcf45 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:05.729912 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a162a-2623-4831-9b73-e8f33fcdcf45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:05.731823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a162a-2623-4831-9b73-e8f33fcdcf45 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:05.742874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1378/2746] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:52:05.746783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eb53a-dcc0-4ab3-9f68-54d672b77f08 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:05.757778 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eb53a-dcc0-4ab3-9f68-54d672b77f08 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:05.762536 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345ac55-60d4-4358-a050-e49c0199a01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:05.763193 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345ac55-60d4-4358-a050-e49c0199a01f admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:05.763472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345ac55-60d4-4358-a050-e49c0199a01f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:05.763745 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345ac55-60d4-4358-a050-e49c0199a01f admin admin] PATH_INFO: `/v3/projects/bd7cf7a2010f419688871e4c9469084d/users/38358c71d0eb4d63a008925b18e68adb/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:05.779286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345ac55-60d4-4358-a050-e49c0199a01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7cf7a2010f419688871e4c9469084d, user_id=38358c71d0eb4d63a008925b18e68adb, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:05.782829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345ac55-60d4-4358-a050-e49c0199a01f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:05.892951 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1369/2747] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:52:05 2022] PUT /identity/v3/projects/bd7cf7a2010f419688871e4c9469084d/users/38358c71d0eb4d63a008925b18e68adb/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:52:05.897887 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d75ab-200f-4331-b477-9f14b716acf4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:05.898207 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d75ab-200f-4331-b477-9f14b716acf4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:05.898485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d75ab-200f-4331-b477-9f14b716acf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:05.917724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-4c9d75ab-200f-4331-b477-9f14b716acf4 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:52:05.919076 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4c9d75ab-200f-4331-b477-9f14b716acf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:05.981863 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1379/2748] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:52:05.992229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345ac55-60d4-4358-a050-e49c0199a01f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:05.994079 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e345ac55-60d4-4358-a050-e49c0199a01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:06.019500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:52:06.056714 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345ac55-60d4-4358-a050-e49c0199a01f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:06.060448 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56630bd1-3307-438f-87ef-6ed7a180f5d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:06.061064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56630bd1-3307-438f-87ef-6ed7a180f5d3 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:06.061346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56630bd1-3307-438f-87ef-6ed7a180f5d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:06.061618 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56630bd1-3307-438f-87ef-6ed7a180f5d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:06.071682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56630bd1-3307-438f-87ef-6ed7a180f5d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:06.073167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56630bd1-3307-438f-87ef-6ed7a180f5d3 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:06.083729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1370/2749] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:52:12.200458 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611aa25-71df-4a22-8db7-9d148001fa62 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:12.201094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611aa25-71df-4a22-8db7-9d148001fa62 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:12.201094 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611aa25-71df-4a22-8db7-9d148001fa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:12.230688 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-4611aa25-71df-4a22-8db7-9d148001fa62 None None] MFA Rules not processed for user `38358c71d0eb4d63a008925b18e68adb`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:52:12.232016 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4611aa25-71df-4a22-8db7-9d148001fa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:12.250880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:52:12.311178 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1380/2750] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:52:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:52:12.311350 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56630bd1-3307-438f-87ef-6ed7a180f5d3 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:12.317313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56630bd1-3307-438f-87ef-6ed7a180f5d3 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:12.322103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df94117-cc86-4734-863b-262d3781fa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:12.322729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df94117-cc86-4734-863b-262d3781fa2a admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:12.323006 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df94117-cc86-4734-863b-262d3781fa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:12.323281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df94117-cc86-4734-863b-262d3781fa2a admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:12.324017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df94117-cc86-4734-863b-262d3781fa2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:12.326190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df94117-cc86-4734-863b-262d3781fa2a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:12.435780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1371/2751] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:52:12 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:52:12.440031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611aa25-71df-4a22-8db7-9d148001fa62 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:12.450810 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611aa25-71df-4a22-8db7-9d148001fa62 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:12.507915 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c2e23-e931-429b-9ab5-71cc58d6454b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:12.508531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c2e23-e931-429b-9ab5-71cc58d6454b admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:12.508812 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c2e23-e931-429b-9ab5-71cc58d6454b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:12.509085 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c2e23-e931-429b-9ab5-71cc58d6454b admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:12.509750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c2e23-e931-429b-9ab5-71cc58d6454b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:12.511717 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c2e23-e931-429b-9ab5-71cc58d6454b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:12.651900 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1381/2752] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:52:12 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:52:12.655811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df94117-cc86-4734-863b-262d3781fa2a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:12.666450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df94117-cc86-4734-863b-262d3781fa2a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:12.670936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dcc36-4982-4ac8-9a4b-ec6b2f5a41c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:12.671538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dcc36-4982-4ac8-9a4b-ec6b2f5a41c2 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:12.671867 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dcc36-4982-4ac8-9a4b-ec6b2f5a41c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:12.672226 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dcc36-4982-4ac8-9a4b-ec6b2f5a41c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:12.672911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dcc36-4982-4ac8-9a4b-ec6b2f5a41c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:12.674537 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dcc36-4982-4ac8-9a4b-ec6b2f5a41c2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:12.684990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1372/2753] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:52:12.690376 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c2e23-e931-429b-9ab5-71cc58d6454b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:12.700647 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c2e23-e931-429b-9ab5-71cc58d6454b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:12.706586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7e9ab-4456-44fc-b123-1dcc03d445ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:12.707263 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7e9ab-4456-44fc-b123-1dcc03d445ea admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:12.707546 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7e9ab-4456-44fc-b123-1dcc03d445ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:12.707824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7e9ab-4456-44fc-b123-1dcc03d445ea admin admin] PATH_INFO: `/v3/projects/628dc286f6eb44888f5b4892cd53b945/users/8628ae1386f94e4cb3c47e8b207be4a1/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:12.722565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7e9ab-4456-44fc-b123-1dcc03d445ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=628dc286f6eb44888f5b4892cd53b945, user_id=8628ae1386f94e4cb3c47e8b207be4a1, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:12.725243 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7e9ab-4456-44fc-b123-1dcc03d445ea admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:12.800674 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1382/2754] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:52:12 2022] PUT /identity/v3/projects/628dc286f6eb44888f5b4892cd53b945/users/8628ae1386f94e4cb3c47e8b207be4a1/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:52:12.804657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dcc36-4982-4ac8-9a4b-ec6b2f5a41c2 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:12.815801 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dcc36-4982-4ac8-9a4b-ec6b2f5a41c2 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:12.873387 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4bcef-9b24-4e2d-bfac-21a3d36a0be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:12.874055 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4bcef-9b24-4e2d-bfac-21a3d36a0be7 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:12.879890 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4bcef-9b24-4e2d-bfac-21a3d36a0be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:12.879890 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4bcef-9b24-4e2d-bfac-21a3d36a0be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:12.879890 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4bcef-9b24-4e2d-bfac-21a3d36a0be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:12.879890 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4bcef-9b24-4e2d-bfac-21a3d36a0be7 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:12.887492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1373/2755] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:52:12.892799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7e9ab-4456-44fc-b123-1dcc03d445ea admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:12.904442 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7e9ab-4456-44fc-b123-1dcc03d445ea admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:12.909468 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aaf2b-cafc-47e6-9999-87a9d3b4aa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:12.918673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aaf2b-cafc-47e6-9999-87a9d3b4aa19 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:12.918673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aaf2b-cafc-47e6-9999-87a9d3b4aa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:12.918673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aaf2b-cafc-47e6-9999-87a9d3b4aa19 admin admin] PATH_INFO: `/v3/projects/628dc286f6eb44888f5b4892cd53b945/users/8628ae1386f94e4cb3c47e8b207be4a1/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:12.918673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4aaf2b-cafc-47e6-9999-87a9d3b4aa19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=628dc286f6eb44888f5b4892cd53b945, user_id=8628ae1386f94e4cb3c47e8b207be4a1, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:12.918673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4aaf2b-cafc-47e6-9999-87a9d3b4aa19 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:12.927132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1383/2756] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:52:12 2022] PUT /identity/v3/projects/628dc286f6eb44888f5b4892cd53b945/users/8628ae1386f94e4cb3c47e8b207be4a1/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:52:18.586479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f595e0-7981-4c26-a281-5dc9251f8168 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:18.586994 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f595e0-7981-4c26-a281-5dc9251f8168 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:18.587120 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f595e0-7981-4c26-a281-5dc9251f8168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:18.616785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-70f595e0-7981-4c26-a281-5dc9251f8168 None None] MFA Rules not processed for user `8628ae1386f94e4cb3c47e8b207be4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:52:18.618527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-70f595e0-7981-4c26-a281-5dc9251f8168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:18.725133 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1374/2757] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:52:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:52:18.739155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aaf2b-cafc-47e6-9999-87a9d3b4aa19 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:18.741042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-8f4aaf2b-cafc-47e6-9999-87a9d3b4aa19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:18.826797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aaf2b-cafc-47e6-9999-87a9d3b4aa19 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:18.833195 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00059285-eea9-4ca2-a774-099bcacfee66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:18.833824 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00059285-eea9-4ca2-a774-099bcacfee66 service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:18.834136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00059285-eea9-4ca2-a774-099bcacfee66 service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:18.834412 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00059285-eea9-4ca2-a774-099bcacfee66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:18.840050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:52:18.893702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00059285-eea9-4ca2-a774-099bcacfee66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:18.895404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00059285-eea9-4ca2-a774-099bcacfee66 service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:18.905512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1384/2758] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:52:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:52:21.544949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b28c-677f-4581-abfd-ac3cc51b5946 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:21.545505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b28c-677f-4581-abfd-ac3cc51b5946 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:21.545728 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b28c-677f-4581-abfd-ac3cc51b5946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:21.568437 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-ce55b28c-677f-4581-abfd-ac3cc51b5946 None None] MFA Rules not processed for user `38358c71d0eb4d63a008925b18e68adb`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:52:21.569810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ce55b28c-677f-4581-abfd-ac3cc51b5946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:21.579222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1375/2759] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:52:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:52:28.914178 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=24 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:52:31.561355 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:52:45.885325 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00059285-eea9-4ca2-a774-099bcacfee66 service nova] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:52:45.887687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-00059285-eea9-4ca2-a774-099bcacfee66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:45.982911 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00059285-eea9-4ca2-a774-099bcacfee66 service nova] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:52:45.988944 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2723845-2661-43a0-a6f6-b01797b74de4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:52:45.989616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2723845-2661-43a0-a6f6-b01797b74de4 service placement] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:52:45.989943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2723845-2661-43a0-a6f6-b01797b74de4 service placement] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:52:45.990250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2723845-2661-43a0-a6f6-b01797b74de4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:52:45.992109 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b2723845-2661-43a0-a6f6-b01797b74de4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:52:46.005439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2723845-2661-43a0-a6f6-b01797b74de4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:52:46.007051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2723845-2661-43a0-a6f6-b01797b74de4 service placement] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:52:46.019110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1385/2760] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:52:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:52:56.013867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=32 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:53:01.484967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b28c-677f-4581-abfd-ac3cc51b5946 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:53:01.499364 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b28c-677f-4581-abfd-ac3cc51b5946 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:53:01.504856 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711a930-43c4-43aa-aaff-7e42e4ee5a32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:53:01.505468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711a930-43c4-43aa-aaff-7e42e4ee5a32 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:01.505757 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711a930-43c4-43aa-aaff-7e42e4ee5a32 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:01.506060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711a930-43c4-43aa-aaff-7e42e4ee5a32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:01.507919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c711a930-43c4-43aa-aaff-7e42e4ee5a32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:01.521077 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711a930-43c4-43aa-aaff-7e42e4ee5a32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:53:01.522727 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711a930-43c4-43aa-aaff-7e42e4ee5a32 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:53:01.534665 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1376/2761] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:53:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:53:04.973632 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2723845-2661-43a0-a6f6-b01797b74de4 service placement] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:53:04.975061 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-b2723845-2661-43a0-a6f6-b01797b74de4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:05.033012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2723845-2661-43a0-a6f6-b01797b74de4 service placement] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:53:05.036622 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7938202-f506-4cc2-80a8-20243f9b057c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:53:05.037241 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7938202-f506-4cc2-80a8-20243f9b057c service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:05.037523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7938202-f506-4cc2-80a8-20243f9b057c service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:05.037799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7938202-f506-4cc2-80a8-20243f9b057c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:05.047184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7938202-f506-4cc2-80a8-20243f9b057c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:53:05.048600 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7938202-f506-4cc2-80a8-20243f9b057c service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:53:05.058233 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1386/2762] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:53:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:53:11.529677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=6 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:53:15.053906 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=26 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:53:17.842258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711a930-43c4-43aa-aaff-7e42e4ee5a32 service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:53:17.843758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c711a930-43c4-43aa-aaff-7e42e4ee5a32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:17.858865 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711a930-43c4-43aa-aaff-7e42e4ee5a32 service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:53:17.867677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f470f-50ec-436f-ae14-55d712033efb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:53:17.868297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f470f-50ec-436f-ae14-55d712033efb service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:17.868578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f470f-50ec-436f-ae14-55d712033efb service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:17.868849 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f470f-50ec-436f-ae14-55d712033efb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:17.870784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1e4f470f-50ec-436f-ae14-55d712033efb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:17.886225 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f470f-50ec-436f-ae14-55d712033efb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:53:17.888045 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f470f-50ec-436f-ae14-55d712033efb service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:53:17.901304 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1377/2763] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:53:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:53:26.954239 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084454a4-597d-4064-8ebb-2fd2891aac67 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:26.954742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084454a4-597d-4064-8ebb-2fd2891aac67 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:26.954851 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084454a4-597d-4064-8ebb-2fd2891aac67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:27.006039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-084454a4-597d-4064-8ebb-2fd2891aac67 None None] MFA Rules not processed for user `6027cfb8625c4d028e7160a85e81bb74`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:53:27.007259 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-084454a4-597d-4064-8ebb-2fd2891aac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:27.070268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1387/2764] 10.222.0.73 () {58 vars in 967 bytes} [Thu Aug 4 21:53:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:53:27.207767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f470f-50ec-436f-ae14-55d712033efb service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:53:27.210112 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1e4f470f-50ec-436f-ae14-55d712033efb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:27.302391 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f470f-50ec-436f-ae14-55d712033efb service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:53:27.307423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206de115-280a-41b0-920c-555e4aaf69b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3342e1039131496ba121b04685ec7b3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:53:27.309842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206de115-280a-41b0-920c-555e4aaf69b2 service swift] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:27.309842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206de115-280a-41b0-920c-555e4aaf69b2 service swift] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:27.309842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206de115-280a-41b0-920c-555e4aaf69b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:27.316696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206de115-280a-41b0-920c-555e4aaf69b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:53:27.318917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206de115-280a-41b0-920c-555e4aaf69b2 service swift] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:53:27.326963 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1378/2765] 10.222.0.73 () {60 vars in 1384 bytes} [Thu Aug 4 21:53:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:53:32.086083 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084454a4-597d-4064-8ebb-2fd2891aac67 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:53:32.098929 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084454a4-597d-4064-8ebb-2fd2891aac67 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:53:32.106785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db8152-d788-4eb7-9afb-bae856b2d89f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:53:32.106785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db8152-d788-4eb7-9afb-bae856b2d89f service placement] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:32.106785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db8152-d788-4eb7-9afb-bae856b2d89f service placement] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:32.106785 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db8152-d788-4eb7-9afb-bae856b2d89f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:32.108423 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-71db8152-d788-4eb7-9afb-bae856b2d89f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:32.122145 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db8152-d788-4eb7-9afb-bae856b2d89f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:53:32.126558 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db8152-d788-4eb7-9afb-bae856b2d89f service placement] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:53:32.138995 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1388/2766] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:53:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:53:36.021552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206de115-280a-41b0-920c-555e4aaf69b2 service swift] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:53:36.023075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-206de115-280a-41b0-920c-555e4aaf69b2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:36.092633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206de115-280a-41b0-920c-555e4aaf69b2 service swift] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:53:36.097477 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:53:36.098114 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:36.098488 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:36.098845 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] PATH_INFO: `/v3/users/11ce89a0a9254498977b9ef531aa3b2e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:36.099997 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11ce89a0a9254498977b9ef531aa3b2e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:53:36.101892 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:53:36.275763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11ce89a0a9254498977b9ef531aa3b2e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:36.375026 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11ce89a0a9254498977b9ef531aa3b2e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:36.375592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11ce89a0a9254498977b9ef531aa3b2e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:36.380657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:53:36.388662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11ce89a0a9254498977b9ef531aa3b2e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:36.395862 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11ce89a0a9254498977b9ef531aa3b2e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:36.401702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1379/2767] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:53:36 2022] DELETE /identity/v3/users/11ce89a0a9254498977b9ef531aa3b2e => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:53:36.657147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db8152-d788-4eb7-9afb-bae856b2d89f service placement] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:53:36.669952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db8152-d788-4eb7-9afb-bae856b2d89f service placement] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:53:36.728616 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:53:36.729253 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:36.729533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:36.729804 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] PATH_INFO: `/v3/projects/b1685a61dd0844269efb39407dcd1e67` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:36.730971 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1685a61dd0844269efb39407dcd1e67)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:53:36.732991 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:53:36.867267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1685a61dd0844269efb39407dcd1e67', 'request_id': 'req-107a568f-1bd4-45ee-9b01-13045892c68c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:36.929428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1685a61dd0844269efb39407dcd1e67', 'request_id': 'req-107a568f-1bd4-45ee-9b01-13045892c68c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:36.933779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:53:36.936088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1685a61dd0844269efb39407dcd1e67', 'request_id': 'req-107a568f-1bd4-45ee-9b01-13045892c68c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:36.947817 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] The token cache is being invalidate because project b1685a61dd0844269efb39407dcd1e67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:53:36.948088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:36.948783 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:36.950326 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1389/2768] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:53:36 2022] DELETE /identity/v3/projects/b1685a61dd0844269efb39407dcd1e67 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:53:36.954856 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:53:36.956568 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:36.976287 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ee571-105a-4da4-a46e-6c2ef9e30cc6 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:53:37.033320 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf77feb-5dcd-4144-926e-974d4a98fef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:53:37.034048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf77feb-5dcd-4144-926e-974d4a98fef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:37.034268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf77feb-5dcd-4144-926e-974d4a98fef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:37.034540 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf77feb-5dcd-4144-926e-974d4a98fef5 admin admin] PATH_INFO: `/v3/users/b452be2babca422aa39e5763dc6b70c2` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:37.035730 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf77feb-5dcd-4144-926e-974d4a98fef5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b452be2babca422aa39e5763dc6b70c2)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:53:37.037642 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf77feb-5dcd-4144-926e-974d4a98fef5 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:53:37.211534 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-acf77feb-5dcd-4144-926e-974d4a98fef5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b452be2babca422aa39e5763dc6b70c2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:37.288329 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-acf77feb-5dcd-4144-926e-974d4a98fef5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b452be2babca422aa39e5763dc6b70c2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:37.288974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-acf77feb-5dcd-4144-926e-974d4a98fef5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b452be2babca422aa39e5763dc6b70c2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:37.303409 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-acf77feb-5dcd-4144-926e-974d4a98fef5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b452be2babca422aa39e5763dc6b70c2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:37.311302 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-acf77feb-5dcd-4144-926e-974d4a98fef5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b452be2babca422aa39e5763dc6b70c2'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:37.318842 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1380/2769] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:53:36 2022] DELETE /identity/v3/users/b452be2babca422aa39e5763dc6b70c2 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:53:37.554592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:53:37.556172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:37.569285 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a568f-1bd4-45ee-9b01-13045892c68c admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:53:37.634636 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:53:37.635291 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:37.635573 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:37.637686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] PATH_INFO: `/v3/projects/087cae9f6dda4ca5be96f9e185082e3a` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:37.637686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=087cae9f6dda4ca5be96f9e185082e3a)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:53:37.639062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:53:37.735184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087cae9f6dda4ca5be96f9e185082e3a', 'request_id': 'req-1897d215-c916-46e2-b80e-87c709241c1f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:37.781105 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087cae9f6dda4ca5be96f9e185082e3a', 'request_id': 'req-1897d215-c916-46e2-b80e-87c709241c1f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:37.787880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087cae9f6dda4ca5be96f9e185082e3a', 'request_id': 'req-1897d215-c916-46e2-b80e-87c709241c1f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:37.802623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] The token cache is being invalidate because project 087cae9f6dda4ca5be96f9e185082e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:53:37.802960 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:37.803689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:53:37.805319 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1390/2770] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:53:37 2022] DELETE /identity/v3/projects/087cae9f6dda4ca5be96f9e185082e3a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:53:47.314060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:53:47.799072 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=8,keystone:SELECT=55,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:53:51.611743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552de46a-5164-4e99-95d3-b015b4d2d661 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:51.612272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552de46a-5164-4e99-95d3-b015b4d2d661 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:51.612272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552de46a-5164-4e99-95d3-b015b4d2d661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:51.631788 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-552de46a-5164-4e99-95d3-b015b4d2d661 None None] MFA Rules not processed for user `a022ce50f3334e919dcf0008bb6afcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:53:51.632982 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-552de46a-5164-4e99-95d3-b015b4d2d661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:51.686869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1381/2771] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:53:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:53:52.005030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:53:52.006460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:52.063638 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897d215-c916-46e2-b80e-87c709241c1f admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:53:52.067176 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e843e29-d8b5-4715-80a3-8a88e37f7a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:53:52.067788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843e29-d8b5-4715-80a3-8a88e37f7a02 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:53:52.068069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843e29-d8b5-4715-80a3-8a88e37f7a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:53:52.068344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843e29-d8b5-4715-80a3-8a88e37f7a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:53:52.070168 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6e843e29-d8b5-4715-80a3-8a88e37f7a02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:53:52.079671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e843e29-d8b5-4715-80a3-8a88e37f7a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:53:52.081081 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e843e29-d8b5-4715-80a3-8a88e37f7a02 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:53:52.089106 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1391/2772] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:53:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:54:01.679655 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=25 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:54:02.085374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=26 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:54:31.099210 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552de46a-5164-4e99-95d3-b015b4d2d661 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:54:31.101317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-552de46a-5164-4e99-95d3-b015b4d2d661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:54:31.171438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552de46a-5164-4e99-95d3-b015b4d2d661 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:54:31.177067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05b77f-eafb-465c-9f6c-ae2e153d1fd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:54:31.177709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05b77f-eafb-465c-9f6c-ae2e153d1fd9 service placement] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:54:31.177991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05b77f-eafb-465c-9f6c-ae2e153d1fd9 service placement] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:54:31.178295 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05b77f-eafb-465c-9f6c-ae2e153d1fd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:54:31.180086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2b05b77f-eafb-465c-9f6c-ae2e153d1fd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:54:31.191622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05b77f-eafb-465c-9f6c-ae2e153d1fd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:54:31.193197 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05b77f-eafb-465c-9f6c-ae2e153d1fd9 service placement] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:54:31.203230 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1382/2773] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:54:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:54:41.198827 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=32 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:55:22.954074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e843e29-d8b5-4715-80a3-8a88e37f7a02 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:55:22.955734 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6e843e29-d8b5-4715-80a3-8a88e37f7a02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:55:23.026057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e843e29-d8b5-4715-80a3-8a88e37f7a02 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:55:23.030335 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e93a0-ed9c-4ce8-ac9e-03cbd69c463b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:55:23.030963 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e93a0-ed9c-4ce8-ac9e-03cbd69c463b service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:55:23.031245 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e93a0-ed9c-4ce8-ac9e-03cbd69c463b service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:55:23.031520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e93a0-ed9c-4ce8-ac9e-03cbd69c463b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:55:23.033297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-cb6e93a0-ed9c-4ce8-ac9e-03cbd69c463b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:55:23.046598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05b77f-eafb-465c-9f6c-ae2e153d1fd9 service placement] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:55:23.056261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05b77f-eafb-465c-9f6c-ae2e153d1fd9 service placement] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:55:23.060711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127a2d3-9b0f-42e7-8964-24dcd015f8de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:55:23.065167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127a2d3-9b0f-42e7-8964-24dcd015f8de service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:55:23.065167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127a2d3-9b0f-42e7-8964-24dcd015f8de service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:55:23.065167 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127a2d3-9b0f-42e7-8964-24dcd015f8de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:55:23.070174 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c127a2d3-9b0f-42e7-8964-24dcd015f8de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:55:23.119803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e93a0-ed9c-4ce8-ac9e-03cbd69c463b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:55:23.121234 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e93a0-ed9c-4ce8-ac9e-03cbd69c463b service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:55:23.138308 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1392/2774] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:55:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:55:23.164696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127a2d3-9b0f-42e7-8964-24dcd015f8de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:55:23.166375 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127a2d3-9b0f-42e7-8964-24dcd015f8de service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:55:23.183785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1383/2775] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:55:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:55:33.129662 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=54 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:55:33.173048 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=34 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:55:40.450681 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f72c-0d00-4072-bd5e-aac807aa1f95 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:55:40.451266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f72c-0d00-4072-bd5e-aac807aa1f95 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:55:40.451266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f72c-0d00-4072-bd5e-aac807aa1f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:55:40.473127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1040f72c-0d00-4072-bd5e-aac807aa1f95 None None] MFA Rules not processed for user `a022ce50f3334e919dcf0008bb6afcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:55:40.474382 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1040f72c-0d00-4072-bd5e-aac807aa1f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:55:40.481381 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1393/2776] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:55:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:55:43.081528 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127a2d3-9b0f-42e7-8964-24dcd015f8de service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:55:43.091160 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127a2d3-9b0f-42e7-8964-24dcd015f8de service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:55:43.096078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48327d38-582d-407c-aaab-3e214a0688af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:55:43.096826 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48327d38-582d-407c-aaab-3e214a0688af service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:55:43.097181 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48327d38-582d-407c-aaab-3e214a0688af service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:55:43.097468 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48327d38-582d-407c-aaab-3e214a0688af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:55:43.099444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-48327d38-582d-407c-aaab-3e214a0688af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:55:43.117309 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48327d38-582d-407c-aaab-3e214a0688af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:55:43.119792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48327d38-582d-407c-aaab-3e214a0688af service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:55:43.129818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1384/2777] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:55:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:55:43.209996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040f72c-0d00-4072-bd5e-aac807aa1f95 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:55:43.224419 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040f72c-0d00-4072-bd5e-aac807aa1f95 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:55:43.228148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff79fed-2a7a-414c-84d2-5f6342861e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:55:43.228761 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff79fed-2a7a-414c-84d2-5f6342861e23 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:55:43.229041 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff79fed-2a7a-414c-84d2-5f6342861e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:55:43.229313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff79fed-2a7a-414c-84d2-5f6342861e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:55:43.231189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4ff79fed-2a7a-414c-84d2-5f6342861e23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:55:43.245641 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff79fed-2a7a-414c-84d2-5f6342861e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:55:43.247014 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff79fed-2a7a-414c-84d2-5f6342861e23 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:55:43.258899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1394/2778] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:55:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:55:53.126383 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=6 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:55:53.253892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=9 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:57:12.066743 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48327d38-582d-407c-aaab-3e214a0688af service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:12.068441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-48327d38-582d-407c-aaab-3e214a0688af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:12.084797 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48327d38-582d-407c-aaab-3e214a0688af service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:12.090369 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9de3a-fd68-4cc7-b435-acc098b0c615 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:12.090967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9de3a-fd68-4cc7-b435-acc098b0c615 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:12.091246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9de3a-fd68-4cc7-b435-acc098b0c615 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:12.091519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9de3a-fd68-4cc7-b435-acc098b0c615 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:12.093204 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1ea9de3a-fd68-4cc7-b435-acc098b0c615 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:12.104105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9de3a-fd68-4cc7-b435-acc098b0c615 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:12.105719 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9de3a-fd68-4cc7-b435-acc098b0c615 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:12.115262 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1385/2779] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:57:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:57:22.111222 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=6 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:57:29.107512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff79fed-2a7a-414c-84d2-5f6342861e23 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:29.109010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-4ff79fed-2a7a-414c-84d2-5f6342861e23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:29.129956 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff79fed-2a7a-414c-84d2-5f6342861e23 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:29.138916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2b43e-3449-4406-9d62-0bad8c1df8ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:29.139559 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2b43e-3449-4406-9d62-0bad8c1df8ab service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:29.140028 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2b43e-3449-4406-9d62-0bad8c1df8ab service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:29.140474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2b43e-3449-4406-9d62-0bad8c1df8ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:29.143456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-57a2b43e-3449-4406-9d62-0bad8c1df8ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:29.159299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2b43e-3449-4406-9d62-0bad8c1df8ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:29.160855 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2b43e-3449-4406-9d62-0bad8c1df8ab service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:29.173626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1395/2780] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:57:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:57:34.054433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb08d33-d467-4e81-a486-1dbbe3ff581d None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:34.054945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb08d33-d467-4e81-a486-1dbbe3ff581d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:34.055072 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb08d33-d467-4e81-a486-1dbbe3ff581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:34.074985 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-3eb08d33-d467-4e81-a486-1dbbe3ff581d None None] MFA Rules not processed for user `a022ce50f3334e919dcf0008bb6afcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:57:34.076296 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3eb08d33-d467-4e81-a486-1dbbe3ff581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:34.083221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1386/2781] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:57:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:57:37.487305 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2b43e-3449-4406-9d62-0bad8c1df8ab service nova] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:37.488923 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-57a2b43e-3449-4406-9d62-0bad8c1df8ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:37.500826 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2b43e-3449-4406-9d62-0bad8c1df8ab service nova] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:37.504506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d23265-f808-40ac-9b34-61f59e2578f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:37.505131 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d23265-f808-40ac-9b34-61f59e2578f8 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:37.505414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d23265-f808-40ac-9b34-61f59e2578f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:37.505689 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d23265-f808-40ac-9b34-61f59e2578f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:37.507663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-12d23265-f808-40ac-9b34-61f59e2578f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:37.571297 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d23265-f808-40ac-9b34-61f59e2578f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:37.572764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d23265-f808-40ac-9b34-61f59e2578f8 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:37.582565 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1396/2782] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:57:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:57:37.890094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb08d33-d467-4e81-a486-1dbbe3ff581d None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:37.902147 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb08d33-d467-4e81-a486-1dbbe3ff581d None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:37.906910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81366734-8326-4c78-82ec-e8fd68aa8067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:37.907738 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81366734-8326-4c78-82ec-e8fd68aa8067 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:37.908186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81366734-8326-4c78-82ec-e8fd68aa8067 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:37.908681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81366734-8326-4c78-82ec-e8fd68aa8067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:37.910696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-81366734-8326-4c78-82ec-e8fd68aa8067 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:37.921853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81366734-8326-4c78-82ec-e8fd68aa8067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:37.923527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81366734-8326-4c78-82ec-e8fd68aa8067 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:37.933192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1387/2783] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:57:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:57:46.250044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d23265-f808-40ac-9b34-61f59e2578f8 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:46.258967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d23265-f808-40ac-9b34-61f59e2578f8 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:46.262691 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f632b0-75e4-4a50-af4b-0f7034040bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:46.263467 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f632b0-75e4-4a50-af4b-0f7034040bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:46.263697 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f632b0-75e4-4a50-af4b-0f7034040bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:46.263974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f632b0-75e4-4a50-af4b-0f7034040bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:46.265775 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-19f632b0-75e4-4a50-af4b-0f7034040bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:46.328738 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f632b0-75e4-4a50-af4b-0f7034040bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:46.337179 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f632b0-75e4-4a50-af4b-0f7034040bc6 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:46.339543 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1397/2784] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:57:47.928561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=9 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:57:50.655598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81366734-8326-4c78-82ec-e8fd68aa8067 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:50.658393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-81366734-8326-4c78-82ec-e8fd68aa8067 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:50.692688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81366734-8326-4c78-82ec-e8fd68aa8067 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:50.698574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e1226-a2ae-40a4-97f4-8acef50cd2dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:50.700523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e1226-a2ae-40a4-97f4-8acef50cd2dc service placement] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:50.700812 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e1226-a2ae-40a4-97f4-8acef50cd2dc service placement] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:50.701258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e1226-a2ae-40a4-97f4-8acef50cd2dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:50.703322 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-063e1226-a2ae-40a4-97f4-8acef50cd2dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:50.717581 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e1226-a2ae-40a4-97f4-8acef50cd2dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:50.719236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e1226-a2ae-40a4-97f4-8acef50cd2dc service placement] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:50.730297 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1388/2785] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:57:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:57:54.723493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f632b0-75e4-4a50-af4b-0f7034040bc6 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:54.725029 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-19f632b0-75e4-4a50-af4b-0f7034040bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:54.732446 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=65 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:57:54.737664 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f632b0-75e4-4a50-af4b-0f7034040bc6 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:54.742411 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:54.743050 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:54.743334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:54.743611 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] PATH_INFO: `/v3/users/38358c71d0eb4d63a008925b18e68adb` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:54.744789 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38358c71d0eb4d63a008925b18e68adb)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:54.746740 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:55.012367 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38358c71d0eb4d63a008925b18e68adb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:55.098189 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38358c71d0eb4d63a008925b18e68adb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:55.110964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38358c71d0eb4d63a008925b18e68adb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:55.112515 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38358c71d0eb4d63a008925b18e68adb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:55.119186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38358c71d0eb4d63a008925b18e68adb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:55.126520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1398/2786] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:57:54 2022] DELETE /identity/v3/users/38358c71d0eb4d63a008925b18e68adb => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:57:55.397647 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e1226-a2ae-40a4-97f4-8acef50cd2dc service placement] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:55.410447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e1226-a2ae-40a4-97f4-8acef50cd2dc service placement] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:55.474263 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:55.474896 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:55.475175 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:55.475444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] PATH_INFO: `/v3/projects/bd7cf7a2010f419688871e4c9469084d` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:55.476711 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd7cf7a2010f419688871e4c9469084d)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:55.478646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:55.607078 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd7cf7a2010f419688871e4c9469084d', 'request_id': 'req-6eca3725-92a6-46fb-80c5-23d51edc8413'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:55.613425 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd7cf7a2010f419688871e4c9469084d', 'request_id': 'req-6eca3725-92a6-46fb-80c5-23d51edc8413'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:55.709577 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd7cf7a2010f419688871e4c9469084d', 'request_id': 'req-6eca3725-92a6-46fb-80c5-23d51edc8413'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:55.720590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] The token cache is being invalidate because project bd7cf7a2010f419688871e4c9469084d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:57:55.720864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:55.721496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:55.723409 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1389/2787] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:57:55 2022] DELETE /identity/v3/projects/bd7cf7a2010f419688871e4c9469084d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:57:55.727580 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:55.729359 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:55.743466 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac8f7b-1e7c-4a8a-a917-32bd039f7dd4 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:55.805658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72a093-7df9-4e98-9c3d-9f2505eb0d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:55.806354 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72a093-7df9-4e98-9c3d-9f2505eb0d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:55.806635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72a093-7df9-4e98-9c3d-9f2505eb0d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:55.806903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72a093-7df9-4e98-9c3d-9f2505eb0d32 admin admin] PATH_INFO: `/v3/users/8628ae1386f94e4cb3c47e8b207be4a1` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:55.808230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72a093-7df9-4e98-9c3d-9f2505eb0d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8628ae1386f94e4cb3c47e8b207be4a1)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:55.810694 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72a093-7df9-4e98-9c3d-9f2505eb0d32 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:55.977026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3e72a093-7df9-4e98-9c3d-9f2505eb0d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8628ae1386f94e4cb3c47e8b207be4a1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:56.060159 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3e72a093-7df9-4e98-9c3d-9f2505eb0d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8628ae1386f94e4cb3c47e8b207be4a1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:56.072435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3e72a093-7df9-4e98-9c3d-9f2505eb0d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8628ae1386f94e4cb3c47e8b207be4a1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:56.073077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3e72a093-7df9-4e98-9c3d-9f2505eb0d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8628ae1386f94e4cb3c47e8b207be4a1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:56.078214 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:57:56.080977 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-3e72a093-7df9-4e98-9c3d-9f2505eb0d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8628ae1386f94e4cb3c47e8b207be4a1'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:56.086832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1399/2788] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:57:55 2022] DELETE /identity/v3/users/8628ae1386f94e4cb3c47e8b207be4a1 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:57:56.376885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:56.376885 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:56.390593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca3725-92a6-46fb-80c5-23d51edc8413 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:56.450104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:56.451042 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:56.451338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:56.451613 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] PATH_INFO: `/v3/projects/628dc286f6eb44888f5b4892cd53b945` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:56.452783 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] RBAC: Authorizing `identity:delete_project(project_id=628dc286f6eb44888f5b4892cd53b945)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:56.454735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:56.477949 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:57:56.569780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '628dc286f6eb44888f5b4892cd53b945', 'request_id': 'req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:56.576073 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '628dc286f6eb44888f5b4892cd53b945', 'request_id': 'req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:56.647851 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '628dc286f6eb44888f5b4892cd53b945', 'request_id': 'req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:56.661442 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] The token cache is being invalidate because project 628dc286f6eb44888f5b4892cd53b945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:57:56.661740 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:56.662426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-2f3a6ea6-f74c-48b8-acc4-56f4d4ece09e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:57:56.663940 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1390/2789] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:57:56 2022] DELETE /identity/v3/projects/628dc286f6eb44888f5b4892cd53b945 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:57:56.669019 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89e8be-ee21-4714-9bdd-3aa9f8daf29f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:56.669309 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89e8be-ee21-4714-9bdd-3aa9f8daf29f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:56.670313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89e8be-ee21-4714-9bdd-3aa9f8daf29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:56.690342 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ea89e8be-ee21-4714-9bdd-3aa9f8daf29f None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:57:56.691574 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ea89e8be-ee21-4714-9bdd-3aa9f8daf29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:56.750516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1400/2790] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:57:56.758479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353431b-5bd0-40d8-8f3d-af85df777eba None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:56.759370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353431b-5bd0-40d8-8f3d-af85df777eba None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:56.759370 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353431b-5bd0-40d8-8f3d-af85df777eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:56.780138 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-3353431b-5bd0-40d8-8f3d-af85df777eba None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:57:56.781367 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3353431b-5bd0-40d8-8f3d-af85df777eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:56.789244 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1391/2791] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:57:56.793434 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89e8be-ee21-4714-9bdd-3aa9f8daf29f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:56.803650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89e8be-ee21-4714-9bdd-3aa9f8daf29f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:56.808471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a65cf-9a51-4ad7-a696-e38bc4cd34fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:56.809068 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a65cf-9a51-4ad7-a696-e38bc4cd34fd admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:56.809349 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a65cf-9a51-4ad7-a696-e38bc4cd34fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:56.809628 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a65cf-9a51-4ad7-a696-e38bc4cd34fd admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:56.810610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a65cf-9a51-4ad7-a696-e38bc4cd34fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:57:56.810968 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a65cf-9a51-4ad7-a696-e38bc4cd34fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:56.812546 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a65cf-9a51-4ad7-a696-e38bc4cd34fd admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:56.823996 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1401/2792] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:57:56.829164 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353431b-5bd0-40d8-8f3d-af85df777eba None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:56.838414 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353431b-5bd0-40d8-8f3d-af85df777eba None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:56.844024 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0e65b-2184-410e-888c-ba59a2717e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:56.844634 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0e65b-2184-410e-888c-ba59a2717e1f admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:56.844917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0e65b-2184-410e-888c-ba59a2717e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:56.845202 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0e65b-2184-410e-888c-ba59a2717e1f admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:56.845868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0e65b-2184-410e-888c-ba59a2717e1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:56.847802 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0e65b-2184-410e-888c-ba59a2717e1f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:56.942808 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1392/2793] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:57:56 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:57:56.946988 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a65cf-9a51-4ad7-a696-e38bc4cd34fd admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:56.957613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a65cf-9a51-4ad7-a696-e38bc4cd34fd admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:57.014777 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ac512-063b-4d08-9b25-e49a47378c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:57.015395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac512-063b-4d08-9b25-e49a47378c9b admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:57.015676 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac512-063b-4d08-9b25-e49a47378c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:57.015950 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac512-063b-4d08-9b25-e49a47378c9b admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:57.016618 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ac512-063b-4d08-9b25-e49a47378c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:57.018560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ac512-063b-4d08-9b25-e49a47378c9b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:57.093005 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1402/2794] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:57:56 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:57:57.097142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0e65b-2184-410e-888c-ba59a2717e1f admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:57.107944 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0e65b-2184-410e-888c-ba59a2717e1f admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:57.112822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc023f95-7abf-49ab-beb8-40deca769b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:57.113440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc023f95-7abf-49ab-beb8-40deca769b0d admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:57.113717 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc023f95-7abf-49ab-beb8-40deca769b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:57.113988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc023f95-7abf-49ab-beb8-40deca769b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:57.114705 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc023f95-7abf-49ab-beb8-40deca769b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:57.116286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc023f95-7abf-49ab-beb8-40deca769b0d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:57.127363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1393/2795] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:57:57.131490 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ac512-063b-4d08-9b25-e49a47378c9b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:57.142428 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ac512-063b-4d08-9b25-e49a47378c9b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:57.146993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35507bf3-e073-4c4e-9465-ffa0f28787ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:57.147596 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35507bf3-e073-4c4e-9465-ffa0f28787ac admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:57.147876 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35507bf3-e073-4c4e-9465-ffa0f28787ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:57.148146 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35507bf3-e073-4c4e-9465-ffa0f28787ac admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:57.148825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35507bf3-e073-4c4e-9465-ffa0f28787ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:57.150564 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35507bf3-e073-4c4e-9465-ffa0f28787ac admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:57.161486 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1403/2796] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:57:57.165595 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc023f95-7abf-49ab-beb8-40deca769b0d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:57.175767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc023f95-7abf-49ab-beb8-40deca769b0d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:57.180246 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1f4f8-400f-429a-9655-16e007ab95d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:57.180891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1f4f8-400f-429a-9655-16e007ab95d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:57.181183 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1f4f8-400f-429a-9655-16e007ab95d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:57.181438 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1f4f8-400f-429a-9655-16e007ab95d8 admin admin] PATH_INFO: `/v3/projects/ee8894a750724fb987a01514be0ab39b/users/292eee6c79214436962643db0a840708/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:57.196750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1f4f8-400f-429a-9655-16e007ab95d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8894a750724fb987a01514be0ab39b, user_id=292eee6c79214436962643db0a840708, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:57.199330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1f4f8-400f-429a-9655-16e007ab95d8 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:57.300903 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1394/2797] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:57:57 2022] PUT /identity/v3/projects/ee8894a750724fb987a01514be0ab39b/users/292eee6c79214436962643db0a840708/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:57:57.311781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a79db-1579-4e6f-87d9-b23428e70c8e None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:57.312072 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a79db-1579-4e6f-87d9-b23428e70c8e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:57.312348 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a79db-1579-4e6f-87d9-b23428e70c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:57.331402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-613a79db-1579-4e6f-87d9-b23428e70c8e None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:57:57.332591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-613a79db-1579-4e6f-87d9-b23428e70c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:57.382712 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:57:57.391766 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1404/2798] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:57:57.401346 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1f4f8-400f-429a-9655-16e007ab95d8 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:57:57.403143 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-b2c1f4f8-400f-429a-9655-16e007ab95d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:57:57.461063 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1f4f8-400f-429a-9655-16e007ab95d8 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:57:57.465523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793950d-035d-4a6e-bc6d-81395b3e2a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:57:57.466198 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793950d-035d-4a6e-bc6d-81395b3e2a45 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:57:57.466593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793950d-035d-4a6e-bc6d-81395b3e2a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:57:57.467110 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793950d-035d-4a6e-bc6d-81395b3e2a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:57:57.476597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793950d-035d-4a6e-bc6d-81395b3e2a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:57:57.478106 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793950d-035d-4a6e-bc6d-81395b3e2a45 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:57:57.488421 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1395/2799] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:58:02.914246 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863c802-29bc-428c-8362-c705344ab921 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:02.914764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863c802-29bc-428c-8362-c705344ab921 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:02.914888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863c802-29bc-428c-8362-c705344ab921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:02.950451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-1863c802-29bc-428c-8362-c705344ab921 None None] MFA Rules not processed for user `292eee6c79214436962643db0a840708`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:58:02.952315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-1863c802-29bc-428c-8362-c705344ab921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:58:03.026389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1405/2800] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:58:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:58:03.029696 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793950d-035d-4a6e-bc6d-81395b3e2a45 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:58:03.039975 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793950d-035d-4a6e-bc6d-81395b3e2a45 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:58:03.045131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0739ff-4ed0-4f37-a358-341fb2fd3a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:58:03.045805 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0739ff-4ed0-4f37-a358-341fb2fd3a73 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:03.046131 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0739ff-4ed0-4f37-a358-341fb2fd3a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:03.046411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0739ff-4ed0-4f37-a358-341fb2fd3a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:03.047188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0739ff-4ed0-4f37-a358-341fb2fd3a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:58:03.049098 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0739ff-4ed0-4f37-a358-341fb2fd3a73 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:58:03.136695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1396/2801] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:58:03 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:58:03.140799 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863c802-29bc-428c-8362-c705344ab921 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:58:03.151983 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863c802-29bc-428c-8362-c705344ab921 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:58:03.216955 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08491e7a-a881-4b3f-b907-bd27217a2eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:58:03.217787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08491e7a-a881-4b3f-b907-bd27217a2eea admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:03.219640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08491e7a-a881-4b3f-b907-bd27217a2eea admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:03.219640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08491e7a-a881-4b3f-b907-bd27217a2eea admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:03.219640 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08491e7a-a881-4b3f-b907-bd27217a2eea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:58:03.221025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08491e7a-a881-4b3f-b907-bd27217a2eea admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:58:03.355506 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1406/2802] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:58:03 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:58:03.360108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0739ff-4ed0-4f37-a358-341fb2fd3a73 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:58:03.371004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0739ff-4ed0-4f37-a358-341fb2fd3a73 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:58:03.376965 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011349be-a189-43f3-946f-0b55a5115ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:58:03.377573 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011349be-a189-43f3-946f-0b55a5115ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:03.377857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011349be-a189-43f3-946f-0b55a5115ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:03.378169 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011349be-a189-43f3-946f-0b55a5115ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:03.378871 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011349be-a189-43f3-946f-0b55a5115ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:58:03.380467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011349be-a189-43f3-946f-0b55a5115ac2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:58:03.391511 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1397/2803] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:58:03.403056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08491e7a-a881-4b3f-b907-bd27217a2eea admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:58:03.413572 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08491e7a-a881-4b3f-b907-bd27217a2eea admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:58:03.418219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a14d9-22ba-499d-8405-d2ac432bb53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:58:03.418874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a14d9-22ba-499d-8405-d2ac432bb53b admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:03.419160 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a14d9-22ba-499d-8405-d2ac432bb53b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:03.420058 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a14d9-22ba-499d-8405-d2ac432bb53b admin admin] PATH_INFO: `/v3/projects/61eb7dd7c41f41cc9cfbb79f6d0c54ed/users/ad049df1895543e8bf9ee5e99bf3de5f/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:03.436941 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a14d9-22ba-499d-8405-d2ac432bb53b admin admin] RBAC: Authorizing `identity:create_grant(project_id=61eb7dd7c41f41cc9cfbb79f6d0c54ed, user_id=ad049df1895543e8bf9ee5e99bf3de5f, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:58:03.440267 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a14d9-22ba-499d-8405-d2ac432bb53b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:58:03.534086 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1407/2804] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:58:03 2022] PUT /identity/v3/projects/61eb7dd7c41f41cc9cfbb79f6d0c54ed/users/ad049df1895543e8bf9ee5e99bf3de5f/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:58:03.538351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011349be-a189-43f3-946f-0b55a5115ac2 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:58:03.549541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011349be-a189-43f3-946f-0b55a5115ac2 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:58:03.559162 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:58:03.606035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c4add-72e9-460a-863a-3f53560c5ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:58:03.606674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c4add-72e9-460a-863a-3f53560c5ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:03.607029 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c4add-72e9-460a-863a-3f53560c5ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:03.607911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c4add-72e9-460a-863a-3f53560c5ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:03.608644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c4add-72e9-460a-863a-3f53560c5ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:58:03.610251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c4add-72e9-460a-863a-3f53560c5ef1 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:58:03.620291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1398/2805] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:58:03.625244 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a14d9-22ba-499d-8405-d2ac432bb53b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:58:03.635659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a14d9-22ba-499d-8405-d2ac432bb53b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:58:03.640381 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19435eeb-fb7d-4fc6-9ad2-d938fea255dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:58:03.641034 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19435eeb-fb7d-4fc6-9ad2-d938fea255dd admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:03.641312 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19435eeb-fb7d-4fc6-9ad2-d938fea255dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:03.641584 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19435eeb-fb7d-4fc6-9ad2-d938fea255dd admin admin] PATH_INFO: `/v3/projects/61eb7dd7c41f41cc9cfbb79f6d0c54ed/users/ad049df1895543e8bf9ee5e99bf3de5f/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:03.643551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19435eeb-fb7d-4fc6-9ad2-d938fea255dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=61eb7dd7c41f41cc9cfbb79f6d0c54ed, user_id=ad049df1895543e8bf9ee5e99bf3de5f, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:58:03.646274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19435eeb-fb7d-4fc6-9ad2-d938fea255dd admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:58:03.657266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1408/2806] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:58:03 2022] PUT /identity/v3/projects/61eb7dd7c41f41cc9cfbb79f6d0c54ed/users/ad049df1895543e8bf9ee5e99bf3de5f/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:58:09.601154 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65adfb-eb9d-4c21-931b-315ebac56293 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:09.601703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65adfb-eb9d-4c21-931b-315ebac56293 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:09.601703 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65adfb-eb9d-4c21-931b-315ebac56293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:09.636823 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1c65adfb-eb9d-4c21-931b-315ebac56293 None None] MFA Rules not processed for user `ad049df1895543e8bf9ee5e99bf3de5f`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:58:09.638881 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1c65adfb-eb9d-4c21-931b-315ebac56293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:58:09.721919 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1399/2807] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:58:09 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:58:09.744951 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19435eeb-fb7d-4fc6-9ad2-d938fea255dd admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:58:09.750466 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-19435eeb-fb7d-4fc6-9ad2-d938fea255dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:58:09.824268 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:58:09.858531 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19435eeb-fb7d-4fc6-9ad2-d938fea255dd admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:58:09.865417 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c1086-0a95-4201-89b5-31c2c8860de9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:58:09.866410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c1086-0a95-4201-89b5-31c2c8860de9 service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:09.866688 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c1086-0a95-4201-89b5-31c2c8860de9 service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:09.866959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c1086-0a95-4201-89b5-31c2c8860de9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:09.936493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c1086-0a95-4201-89b5-31c2c8860de9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:58:09.938443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c1086-0a95-4201-89b5-31c2c8860de9 service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:58:09.950690 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1409/2808] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:58:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:58:19.718771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=57 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:58:20.072100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=37 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:58:35.641186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32e0db-b044-40a9-aa72-90e5beb706ab None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:35.641734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32e0db-b044-40a9-aa72-90e5beb706ab None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:35.641734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32e0db-b044-40a9-aa72-90e5beb706ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:35.670630 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-8c32e0db-b044-40a9-aa72-90e5beb706ab None None] MFA Rules not processed for user `292eee6c79214436962643db0a840708`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:58:35.671837 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8c32e0db-b044-40a9-aa72-90e5beb706ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:58:35.695810 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1400/2809] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:58:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:58:36.033265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c1086-0a95-4201-89b5-31c2c8860de9 service nova] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:58:36.034814 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-972c1086-0a95-4201-89b5-31c2c8860de9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:58:36.108602 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c1086-0a95-4201-89b5-31c2c8860de9 service nova] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:58:36.112822 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31b12e-3809-4815-919e-54887a7b86a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:58:36.113435 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31b12e-3809-4815-919e-54887a7b86a9 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:36.113715 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31b12e-3809-4815-919e-54887a7b86a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:36.113989 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31b12e-3809-4815-919e-54887a7b86a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:36.115759 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7a31b12e-3809-4815-919e-54887a7b86a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:58:36.125526 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31b12e-3809-4815-919e-54887a7b86a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:58:36.127057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31b12e-3809-4815-919e-54887a7b86a9 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:58:36.135127 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1410/2810] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:58:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:58:38.860794 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32e0db-b044-40a9-aa72-90e5beb706ab None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:58:38.863103 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8c32e0db-b044-40a9-aa72-90e5beb706ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:58:38.946741 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32e0db-b044-40a9-aa72-90e5beb706ab None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:58:38.951966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83e4ce-c453-4e36-95d8-3a3213be2db0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:58:38.952597 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83e4ce-c453-4e36-95d8-3a3213be2db0 service placement] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:38.952875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83e4ce-c453-4e36-95d8-3a3213be2db0 service placement] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:38.953148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83e4ce-c453-4e36-95d8-3a3213be2db0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:38.955003 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-7e83e4ce-c453-4e36-95d8-3a3213be2db0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:58:38.966995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83e4ce-c453-4e36-95d8-3a3213be2db0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:58:38.968590 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83e4ce-c453-4e36-95d8-3a3213be2db0 service placement] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:58:38.979708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1401/2811] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:58:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:58:46.131528 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=35 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:58:48.974606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=50 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:58:59.156048 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31b12e-3809-4815-919e-54887a7b86a9 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:58:59.172025 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31b12e-3809-4815-919e-54887a7b86a9 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:58:59.178077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b7e21-9bb9-426c-8efb-711f53c05aa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:58:59.178693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b7e21-9bb9-426c-8efb-711f53c05aa4 service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:58:59.178973 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b7e21-9bb9-426c-8efb-711f53c05aa4 service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:58:59.179242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b7e21-9bb9-426c-8efb-711f53c05aa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:58:59.181031 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-031b7e21-9bb9-426c-8efb-711f53c05aa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:58:59.197102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b7e21-9bb9-426c-8efb-711f53c05aa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:58:59.199269 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b7e21-9bb9-426c-8efb-711f53c05aa4 service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:58:59.211371 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1411/2812] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:58:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:59:02.551667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83e4ce-c453-4e36-95d8-3a3213be2db0 service placement] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:02.570948 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83e4ce-c453-4e36-95d8-3a3213be2db0 service placement] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:02.574508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fc6ca-2dc8-4692-b539-684f8368b7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:02.575108 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fc6ca-2dc8-4692-b539-684f8368b7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:02.575393 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fc6ca-2dc8-4692-b539-684f8368b7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:02.575663 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fc6ca-2dc8-4692-b539-684f8368b7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:02.585153 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833fc6ca-2dc8-4692-b539-684f8368b7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:02.586629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833fc6ca-2dc8-4692-b539-684f8368b7f8 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:02.595915 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1402/2813] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:59:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:59:09.205666 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=6 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:12.591809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=10 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:28.720549 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b7e21-9bb9-426c-8efb-711f53c05aa4 service nova] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:28.729823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b7e21-9bb9-426c-8efb-711f53c05aa4 service nova] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:28.734112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c4675-6773-4afc-a08c-28d8e0fd4d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:28.734708 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c4675-6773-4afc-a08c-28d8e0fd4d9c service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:28.734985 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c4675-6773-4afc-a08c-28d8e0fd4d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:28.735255 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c4675-6773-4afc-a08c-28d8e0fd4d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:28.737012 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-570c4675-6773-4afc-a08c-28d8e0fd4d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:28.819244 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c4675-6773-4afc-a08c-28d8e0fd4d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:28.820717 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c4675-6773-4afc-a08c-28d8e0fd4d9c service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:28.839039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1412/2814] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:59:32.542362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fc6ca-2dc8-4692-b539-684f8368b7f8 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:32.543902 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-833fc6ca-2dc8-4692-b539-684f8368b7f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:32.585074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fc6ca-2dc8-4692-b539-684f8368b7f8 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:32.591080 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983c59d-55a3-4b2c-9155-a4651ce029e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:32.591698 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983c59d-55a3-4b2c-9155-a4651ce029e0 service placement] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:32.591986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983c59d-55a3-4b2c-9155-a4651ce029e0 service placement] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:32.592264 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983c59d-55a3-4b2c-9155-a4651ce029e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:32.594158 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-f983c59d-55a3-4b2c-9155-a4651ce029e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:32.607161 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983c59d-55a3-4b2c-9155-a4651ce029e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:32.608763 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983c59d-55a3-4b2c-9155-a4651ce029e0 service placement] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:32.618677 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1403/2815] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:59:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:59:32.680240 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c4675-6773-4afc-a08c-28d8e0fd4d9c service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:32.681788 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-570c4675-6773-4afc-a08c-28d8e0fd4d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:32.695299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c4675-6773-4afc-a08c-28d8e0fd4d9c service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:32.700845 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c14a42-9062-434e-bb86-03f96b06712e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:32.701497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c14a42-9062-434e-bb86-03f96b06712e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:32.701782 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c14a42-9062-434e-bb86-03f96b06712e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:32.702171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c14a42-9062-434e-bb86-03f96b06712e admin admin] PATH_INFO: `/v3/users/a022ce50f3334e919dcf0008bb6afcf8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:32.704147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c14a42-9062-434e-bb86-03f96b06712e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a022ce50f3334e919dcf0008bb6afcf8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:32.706128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c14a42-9062-434e-bb86-03f96b06712e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:32.905112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-78c14a42-9062-434e-bb86-03f96b06712e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a022ce50f3334e919dcf0008bb6afcf8'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:33.017631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-78c14a42-9062-434e-bb86-03f96b06712e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a022ce50f3334e919dcf0008bb6afcf8'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:33.030108 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-78c14a42-9062-434e-bb86-03f96b06712e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a022ce50f3334e919dcf0008bb6afcf8'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:33.030747 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-78c14a42-9062-434e-bb86-03f96b06712e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a022ce50f3334e919dcf0008bb6afcf8'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:33.037726 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-78c14a42-9062-434e-bb86-03f96b06712e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a022ce50f3334e919dcf0008bb6afcf8'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:33.044374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1413/2816] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:59:32 2022] DELETE /identity/v3/users/a022ce50f3334e919dcf0008bb6afcf8 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:33.326806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983c59d-55a3-4b2c-9155-a4651ce029e0 service placement] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:33.339789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983c59d-55a3-4b2c-9155-a4651ce029e0 service placement] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:33.403366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:33.404017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:33.404304 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:33.404586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] PATH_INFO: `/v3/projects/8c4ab3c7309249228e42c226277fb14f` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:33.405806 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c4ab3c7309249228e42c226277fb14f)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:33.407750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:33.542243 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c4ab3c7309249228e42c226277fb14f', 'request_id': 'req-f0884cbb-a30d-4d38-a8ee-f9f356039263'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:33.549049 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c4ab3c7309249228e42c226277fb14f', 'request_id': 'req-f0884cbb-a30d-4d38-a8ee-f9f356039263'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:33.612076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c4ab3c7309249228e42c226277fb14f', 'request_id': 'req-f0884cbb-a30d-4d38-a8ee-f9f356039263'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:33.624135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] The token cache is being invalidate because project 8c4ab3c7309249228e42c226277fb14f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:59:33.624420 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:33.625043 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-f0884cbb-a30d-4d38-a8ee-f9f356039263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:33.626561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1404/2817] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:59:33 2022] DELETE /identity/v3/projects/8c4ab3c7309249228e42c226277fb14f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:33.631601 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c4b9-31d8-4596-b370-828a3a55e7c2 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:33.631895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c4b9-31d8-4596-b370-828a3a55e7c2 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:33.632170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c4b9-31d8-4596-b370-828a3a55e7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:33.687828 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:33.690594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ad61c4b9-31d8-4596-b370-828a3a55e7c2 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:33.691867 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ad61c4b9-31d8-4596-b370-828a3a55e7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:33.751313 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1414/2818] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:33.760371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfca975-60e0-4e79-bfbb-6d96b331be38 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:33.760668 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfca975-60e0-4e79-bfbb-6d96b331be38 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:33.760946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfca975-60e0-4e79-bfbb-6d96b331be38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:33.781995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-6cfca975-60e0-4e79-bfbb-6d96b331be38 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:33.783399 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6cfca975-60e0-4e79-bfbb-6d96b331be38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:33.792936 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1405/2819] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:33.797671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61c4b9-31d8-4596-b370-828a3a55e7c2 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:33.808964 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61c4b9-31d8-4596-b370-828a3a55e7c2 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:33.813925 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978558d-2dd3-4798-806d-572339941a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:33.814687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978558d-2dd3-4798-806d-572339941a78 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:33.815039 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978558d-2dd3-4798-806d-572339941a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:33.815390 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978558d-2dd3-4798-806d-572339941a78 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:33.816184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978558d-2dd3-4798-806d-572339941a78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:59:33.816553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978558d-2dd3-4798-806d-572339941a78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:33.818299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978558d-2dd3-4798-806d-572339941a78 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:33.831404 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1415/2820] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:59:33.835090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfca975-60e0-4e79-bfbb-6d96b331be38 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:33.844840 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfca975-60e0-4e79-bfbb-6d96b331be38 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:33.849525 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b12852-1cb9-4c4f-a276-16525b0b5edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:33.850180 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b12852-1cb9-4c4f-a276-16525b0b5edc admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:33.850466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b12852-1cb9-4c4f-a276-16525b0b5edc admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:33.850771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b12852-1cb9-4c4f-a276-16525b0b5edc admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:33.851446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b12852-1cb9-4c4f-a276-16525b0b5edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:33.853395 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b12852-1cb9-4c4f-a276-16525b0b5edc admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:33.952735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1406/2821] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:59:33 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:59:33.957253 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978558d-2dd3-4798-806d-572339941a78 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:33.967939 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978558d-2dd3-4798-806d-572339941a78 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:34.025107 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f21a4-e74b-4329-96f3-84cb24f25bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:34.025644 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f21a4-e74b-4329-96f3-84cb24f25bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:34.025927 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f21a4-e74b-4329-96f3-84cb24f25bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:34.026260 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f21a4-e74b-4329-96f3-84cb24f25bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:34.027470 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f21a4-e74b-4329-96f3-84cb24f25bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:34.034046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f21a4-e74b-4329-96f3-84cb24f25bf4 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:34.150134 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1416/2822] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:59:33 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:59:34.154905 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b12852-1cb9-4c4f-a276-16525b0b5edc admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:34.165588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b12852-1cb9-4c4f-a276-16525b0b5edc admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:34.171431 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb1b4d-e6b7-44d1-8e0f-10a0857b6c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:34.172081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb1b4d-e6b7-44d1-8e0f-10a0857b6c55 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:34.172384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb1b4d-e6b7-44d1-8e0f-10a0857b6c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:34.172698 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb1b4d-e6b7-44d1-8e0f-10a0857b6c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:34.173538 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb1b4d-e6b7-44d1-8e0f-10a0857b6c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:34.175275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb1b4d-e6b7-44d1-8e0f-10a0857b6c55 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:34.185748 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1407/2823] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:59:34.189994 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f21a4-e74b-4329-96f3-84cb24f25bf4 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:34.200170 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f21a4-e74b-4329-96f3-84cb24f25bf4 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:34.204821 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca9a06-690e-42fd-a763-87faa0f4d361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:34.205418 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca9a06-690e-42fd-a763-87faa0f4d361 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:34.205702 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca9a06-690e-42fd-a763-87faa0f4d361 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:34.205976 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca9a06-690e-42fd-a763-87faa0f4d361 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:34.206692 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca9a06-690e-42fd-a763-87faa0f4d361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:34.208441 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca9a06-690e-42fd-a763-87faa0f4d361 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:34.219770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1417/2824] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:59:34.228574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb1b4d-e6b7-44d1-8e0f-10a0857b6c55 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:34.239219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb1b4d-e6b7-44d1-8e0f-10a0857b6c55 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:34.243747 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c39b5-2fb7-4901-a232-c742971f240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:34.244426 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c39b5-2fb7-4901-a232-c742971f240a admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:34.244709 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c39b5-2fb7-4901-a232-c742971f240a admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:34.245002 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c39b5-2fb7-4901-a232-c742971f240a admin admin] PATH_INFO: `/v3/projects/a7845a49a73548b9ab451c89c21c91e2/users/217141cdd3a647f78fb99c391a2f431d/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:34.260084 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c39b5-2fb7-4901-a232-c742971f240a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7845a49a73548b9ab451c89c21c91e2, user_id=217141cdd3a647f78fb99c391a2f431d, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:34.263365 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c39b5-2fb7-4901-a232-c742971f240a admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:34.265789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:34.363943 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1408/2825] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:59:34 2022] PUT /identity/v3/projects/a7845a49a73548b9ab451c89c21c91e2/users/217141cdd3a647f78fb99c391a2f431d/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:34.375307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf76db7-ce8e-44f0-aa58-56ccc94ce24e None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:34.375592 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf76db7-ce8e-44f0-aa58-56ccc94ce24e None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:34.377588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf76db7-ce8e-44f0-aa58-56ccc94ce24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:34.403779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-ecf76db7-ce8e-44f0-aa58-56ccc94ce24e None None] MFA Rules not processed for user `217141cdd3a647f78fb99c391a2f431d`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:34.405076 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-ecf76db7-ce8e-44f0-aa58-56ccc94ce24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:34.469205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1418/2826] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:34.481102 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c39b5-2fb7-4901-a232-c742971f240a admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:34.482912 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-2d9c39b5-2fb7-4901-a232-c742971f240a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:34.565864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c39b5-2fb7-4901-a232-c742971f240a admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:34.571641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9d092-8ed8-49e5-9f68-b2e0824ec03d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:34.572271 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9d092-8ed8-49e5-9f68-b2e0824ec03d service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:34.572556 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9d092-8ed8-49e5-9f68-b2e0824ec03d service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:34.572833 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9d092-8ed8-49e5-9f68-b2e0824ec03d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:34.581268 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9d092-8ed8-49e5-9f68-b2e0824ec03d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:34.582921 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9d092-8ed8-49e5-9f68-b2e0824ec03d service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:34.591828 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1409/2827] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:59:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:59:34.805508 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf76db7-ce8e-44f0-aa58-56ccc94ce24e None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:34.809932 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:34.868353 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf76db7-ce8e-44f0-aa58-56ccc94ce24e None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:34.877806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:34.877806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:34.877806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:34.877806 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] PATH_INFO: `/v3/users/217141cdd3a647f78fb99c391a2f431d` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:34.878266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=217141cdd3a647f78fb99c391a2f431d)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:34.878266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:35.083009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '217141cdd3a647f78fb99c391a2f431d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:35.158082 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '217141cdd3a647f78fb99c391a2f431d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:35.170834 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '217141cdd3a647f78fb99c391a2f431d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:35.171499 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '217141cdd3a647f78fb99c391a2f431d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:35.178275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '217141cdd3a647f78fb99c391a2f431d'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:35.184291 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1419/2828] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:59:34 2022] DELETE /identity/v3/users/217141cdd3a647f78fb99c391a2f431d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:35.188687 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec07127-86bb-4c74-9404-9427ed316e90 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:35.189032 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec07127-86bb-4c74-9404-9427ed316e90 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:35.189338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec07127-86bb-4c74-9404-9427ed316e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:35.208724 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1ec07127-86bb-4c74-9404-9427ed316e90 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:35.209956 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1ec07127-86bb-4c74-9404-9427ed316e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:35.268275 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1410/2829] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:35.278827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:35.280450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:35.349886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44e638-af0d-4f27-af77-adad69a8ce39 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:35.353454 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58effd-adb2-46c4-b792-6753897d3166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:35.354108 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58effd-adb2-46c4-b792-6753897d3166 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:35.354384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58effd-adb2-46c4-b792-6753897d3166 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:35.354684 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58effd-adb2-46c4-b792-6753897d3166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:35.363993 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58effd-adb2-46c4-b792-6753897d3166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:35.365426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58effd-adb2-46c4-b792-6753897d3166 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:35.375110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1420/2830] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:59:35.840070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec07127-86bb-4c74-9404-9427ed316e90 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:35.855750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1ec07127-86bb-4c74-9404-9427ed316e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:35.855750 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec07127-86bb-4c74-9404-9427ed316e90 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:35.860035 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:35.860681 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:35.860966 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:35.861236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] PATH_INFO: `/v3/projects/a7845a49a73548b9ab451c89c21c91e2` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:35.862432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7845a49a73548b9ab451c89c21c91e2)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:35.864343 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:35.996505 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7845a49a73548b9ab451c89c21c91e2', 'request_id': 'req-e71027b3-11fe-44c6-ac09-b07d64328e67'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:36.003119 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7845a49a73548b9ab451c89c21c91e2', 'request_id': 'req-e71027b3-11fe-44c6-ac09-b07d64328e67'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:36.079191 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58effd-adb2-46c4-b792-6753897d3166 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:36.080629 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-9c58effd-adb2-46c4-b792-6753897d3166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:36.095338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:36.146874 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7845a49a73548b9ab451c89c21c91e2', 'request_id': 'req-e71027b3-11fe-44c6-ac09-b07d64328e67'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:36.150885 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58effd-adb2-46c4-b792-6753897d3166 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:36.153234 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:36.160216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] The token cache is being invalidate because project a7845a49a73548b9ab451c89c21c91e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:59:36.160695 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:36.161521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-e71027b3-11fe-44c6-ac09-b07d64328e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:36.163338 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1411/2831] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:59:35 2022] DELETE /identity/v3/projects/a7845a49a73548b9ab451c89c21c91e2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:36.169207 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad226f3-3f87-4f0a-b26b-54b70521afa3 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:36.169605 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad226f3-3f87-4f0a-b26b-54b70521afa3 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:36.169983 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad226f3-3f87-4f0a-b26b-54b70521afa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:36.194366 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1ad226f3-3f87-4f0a-b26b-54b70521afa3 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:36.195574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1ad226f3-3f87-4f0a-b26b-54b70521afa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:36.218274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad65a1d-9319-4f27-bbdf-6e1cfe1218d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:36.222298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad65a1d-9319-4f27-bbdf-6e1cfe1218d0 service placement] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:36.222298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad65a1d-9319-4f27-bbdf-6e1cfe1218d0 service placement] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:36.222298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad65a1d-9319-4f27-bbdf-6e1cfe1218d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:36.222298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bad65a1d-9319-4f27-bbdf-6e1cfe1218d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:36.233614 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad65a1d-9319-4f27-bbdf-6e1cfe1218d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:36.235250 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad65a1d-9319-4f27-bbdf-6e1cfe1218d0 service placement] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:36.245671 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1421/2832] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:59:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:59:36.255446 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1412/2833] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:59:36.263670 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9922f5-1901-4dcd-a284-9b8ae690c8c7 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:36.264163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9922f5-1901-4dcd-a284-9b8ae690c8c7 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:36.264516 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9922f5-1901-4dcd-a284-9b8ae690c8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:36.285489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-bd9922f5-1901-4dcd-a284-9b8ae690c8c7 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:36.286722 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bd9922f5-1901-4dcd-a284-9b8ae690c8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:36.295106 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1422/2834] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:36.299440 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad226f3-3f87-4f0a-b26b-54b70521afa3 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:36.309624 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad226f3-3f87-4f0a-b26b-54b70521afa3 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:36.314531 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8db90e-d2b9-48e7-b85f-2f18c7245565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:36.315149 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8db90e-d2b9-48e7-b85f-2f18c7245565 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:36.315447 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8db90e-d2b9-48e7-b85f-2f18c7245565 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:36.315698 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8db90e-d2b9-48e7-b85f-2f18c7245565 admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:36.316384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8db90e-d2b9-48e7-b85f-2f18c7245565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:59:36.316688 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8db90e-d2b9-48e7-b85f-2f18c7245565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:36.318321 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8db90e-d2b9-48e7-b85f-2f18c7245565 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:36.331022 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1413/2835] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:59:36.335407 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9922f5-1901-4dcd-a284-9b8ae690c8c7 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:36.345457 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9922f5-1901-4dcd-a284-9b8ae690c8c7 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:36.350225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52686880-2138-407a-aa5d-862e5db1476b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:36.350838 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686880-2138-407a-aa5d-862e5db1476b admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:36.351120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686880-2138-407a-aa5d-862e5db1476b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:36.351395 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686880-2138-407a-aa5d-862e5db1476b admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:36.352057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52686880-2138-407a-aa5d-862e5db1476b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:36.353934 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52686880-2138-407a-aa5d-862e5db1476b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:36.461087 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1423/2836] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:59:36 2022] POST /identity/v3/projects => generated 407 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:59:36.465555 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8db90e-d2b9-48e7-b85f-2f18c7245565 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:36.475657 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8db90e-d2b9-48e7-b85f-2f18c7245565 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:36.534243 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27370a50-5b90-4c15-9e01-3c4015823d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:36.534957 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27370a50-5b90-4c15-9e01-3c4015823d00 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:36.535341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27370a50-5b90-4c15-9e01-3c4015823d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:36.535619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27370a50-5b90-4c15-9e01-3c4015823d00 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:36.536377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27370a50-5b90-4c15-9e01-3c4015823d00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:36.538311 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27370a50-5b90-4c15-9e01-3c4015823d00 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:36.653838 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1414/2837] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:59:36 2022] POST /identity/v3/users => generated 304 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:59:36.658115 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52686880-2138-407a-aa5d-862e5db1476b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:36.668833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52686880-2138-407a-aa5d-862e5db1476b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:36.673617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cfe7f-c8a5-4312-bd26-770c34b600a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:36.674275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cfe7f-c8a5-4312-bd26-770c34b600a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:36.674588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cfe7f-c8a5-4312-bd26-770c34b600a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:36.674886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cfe7f-c8a5-4312-bd26-770c34b600a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:36.675586 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6cfe7f-c8a5-4312-bd26-770c34b600a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:36.677200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6cfe7f-c8a5-4312-bd26-770c34b600a9 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:36.688605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1424/2838] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:59:36.692868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27370a50-5b90-4c15-9e01-3c4015823d00 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:36.703221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27370a50-5b90-4c15-9e01-3c4015823d00 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:36.708097 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9316a8-b70d-4477-a6f4-b70ce119104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:36.709777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9316a8-b70d-4477-a6f4-b70ce119104f admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:36.709777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9316a8-b70d-4477-a6f4-b70ce119104f admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:36.709777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9316a8-b70d-4477-a6f4-b70ce119104f admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:36.710166 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9316a8-b70d-4477-a6f4-b70ce119104f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:36.711544 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9316a8-b70d-4477-a6f4-b70ce119104f admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:36.723012 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1415/2839] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:59:36.727392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cfe7f-c8a5-4312-bd26-770c34b600a9 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:36.738311 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cfe7f-c8a5-4312-bd26-770c34b600a9 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:36.743140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1c33b-8da8-419c-be6b-d1128b6f74e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:36.743801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1c33b-8da8-419c-be6b-d1128b6f74e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:36.744088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1c33b-8da8-419c-be6b-d1128b6f74e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:36.744361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1c33b-8da8-419c-be6b-d1128b6f74e3 admin admin] PATH_INFO: `/v3/projects/04290bc917cf4f55969f7b6da83ef264/users/96656e603e3748e4847c38f4f8761369/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:36.760743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1c33b-8da8-419c-be6b-d1128b6f74e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04290bc917cf4f55969f7b6da83ef264, user_id=96656e603e3748e4847c38f4f8761369, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:36.763892 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1c33b-8da8-419c-be6b-d1128b6f74e3 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:36.884942 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1425/2840] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:59:36 2022] PUT /identity/v3/projects/04290bc917cf4f55969f7b6da83ef264/users/96656e603e3748e4847c38f4f8761369/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:59:36.896910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c803f-3cb7-495f-b867-c4f81d752cde None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:36.897402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c803f-3cb7-495f-b867-c4f81d752cde None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:36.897502 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c803f-3cb7-495f-b867-c4f81d752cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:36.925729 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-8f1c803f-3cb7-495f-b867-c4f81d752cde None None] MFA Rules not processed for user `96656e603e3748e4847c38f4f8761369`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:36.926975 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-8f1c803f-3cb7-495f-b867-c4f81d752cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:36.975373 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:36.992344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1416/2841] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:36 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:36.996959 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1c33b-8da8-419c-be6b-d1128b6f74e3 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:37.044555 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:37.061140 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1c33b-8da8-419c-be6b-d1128b6f74e3 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:37.065992 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69386d-c4fa-4c0a-92ef-067f5d0bb0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:37.066626 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69386d-c4fa-4c0a-92ef-067f5d0bb0eb admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:37.066903 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69386d-c4fa-4c0a-92ef-067f5d0bb0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:37.067171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69386d-c4fa-4c0a-92ef-067f5d0bb0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:37.067935 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69386d-c4fa-4c0a-92ef-067f5d0bb0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:37.069818 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69386d-c4fa-4c0a-92ef-067f5d0bb0eb admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:37.169036 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1426/2842] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:59:36 2022] POST /identity/v3/projects => generated 409 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:59:37.173559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c803f-3cb7-495f-b867-c4f81d752cde None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:37.185206 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c803f-3cb7-495f-b867-c4f81d752cde None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:37.247371 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdf861-1cf9-4e94-8428-aedda3d70da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:37.248308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdf861-1cf9-4e94-8428-aedda3d70da2 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:37.248567 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdf861-1cf9-4e94-8428-aedda3d70da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:37.248843 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdf861-1cf9-4e94-8428-aedda3d70da2 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:37.249519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdf861-1cf9-4e94-8428-aedda3d70da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:37.251482 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdf861-1cf9-4e94-8428-aedda3d70da2 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:37.337809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1417/2843] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:59:37 2022] POST /identity/v3/users => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:59:37.342533 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69386d-c4fa-4c0a-92ef-067f5d0bb0eb admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:37.353010 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69386d-c4fa-4c0a-92ef-067f5d0bb0eb admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:37.358886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64953805-70c6-429f-b073-e38d23033e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:37.359539 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64953805-70c6-429f-b073-e38d23033e6d admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:37.359825 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64953805-70c6-429f-b073-e38d23033e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:37.360096 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64953805-70c6-429f-b073-e38d23033e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:37.360787 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64953805-70c6-429f-b073-e38d23033e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:37.362414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64953805-70c6-429f-b073-e38d23033e6d admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:37.373129 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1427/2844] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:59:37.377518 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdf861-1cf9-4e94-8428-aedda3d70da2 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:37.388736 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdf861-1cf9-4e94-8428-aedda3d70da2 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:37.393463 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec50346-04e3-4960-bc89-186e1d39c3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:37.394155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50346-04e3-4960-bc89-186e1d39c3d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:37.394441 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50346-04e3-4960-bc89-186e1d39c3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:37.394726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50346-04e3-4960-bc89-186e1d39c3d7 admin admin] PATH_INFO: `/v3/projects/0a3be3ba1cf6426584870abab32177db/users/2fccedd5c6cd4d949d556009fe8f4d37/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:37.410968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec50346-04e3-4960-bc89-186e1d39c3d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3be3ba1cf6426584870abab32177db, user_id=2fccedd5c6cd4d949d556009fe8f4d37, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:37.413633 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec50346-04e3-4960-bc89-186e1d39c3d7 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:37.481593 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1418/2845] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:59:37 2022] PUT /identity/v3/projects/0a3be3ba1cf6426584870abab32177db/users/2fccedd5c6cd4d949d556009fe8f4d37/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:37.485830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64953805-70c6-429f-b073-e38d23033e6d admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:37.496266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64953805-70c6-429f-b073-e38d23033e6d admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:37.556520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f624f33-a3c7-4e0f-b87d-63c5102a78ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:37.559899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f624f33-a3c7-4e0f-b87d-63c5102a78ee admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:37.559899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f624f33-a3c7-4e0f-b87d-63c5102a78ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:37.559899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f624f33-a3c7-4e0f-b87d-63c5102a78ee admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:37.559899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f624f33-a3c7-4e0f-b87d-63c5102a78ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:37.560905 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f624f33-a3c7-4e0f-b87d-63c5102a78ee admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:37.574080 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1428/2846] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:59:37.578767 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec50346-04e3-4960-bc89-186e1d39c3d7 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:37.589888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec50346-04e3-4960-bc89-186e1d39c3d7 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:37.594482 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dc26d-58ee-47d7-be47-03cac7ef5354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:37.595135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dc26d-58ee-47d7-be47-03cac7ef5354 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:37.595419 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dc26d-58ee-47d7-be47-03cac7ef5354 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:37.595690 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dc26d-58ee-47d7-be47-03cac7ef5354 admin admin] PATH_INFO: `/v3/projects/0a3be3ba1cf6426584870abab32177db/users/2fccedd5c6cd4d949d556009fe8f4d37/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:37.597627 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dc26d-58ee-47d7-be47-03cac7ef5354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3be3ba1cf6426584870abab32177db, user_id=2fccedd5c6cd4d949d556009fe8f4d37, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:37.600281 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dc26d-58ee-47d7-be47-03cac7ef5354 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:37.610523 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1419/2847] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:59:37 2022] PUT /identity/v3/projects/0a3be3ba1cf6426584870abab32177db/users/2fccedd5c6cd4d949d556009fe8f4d37/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:37.620895 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a559c55-732d-42cd-81c5-8abdd05aa9a4 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:37.621190 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a559c55-732d-42cd-81c5-8abdd05aa9a4 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:37.621482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a559c55-732d-42cd-81c5-8abdd05aa9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:37.649743 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-0a559c55-732d-42cd-81c5-8abdd05aa9a4 None None] MFA Rules not processed for user `2fccedd5c6cd4d949d556009fe8f4d37`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:37.651112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-0a559c55-732d-42cd-81c5-8abdd05aa9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:37.718413 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1429/2848] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:37 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:37.729754 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dc26d-58ee-47d7-be47-03cac7ef5354 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:37.731558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-842dc26d-58ee-47d7-be47-03cac7ef5354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:37.818363 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dc26d-58ee-47d7-be47-03cac7ef5354 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:37.822945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20419d8c-6609-40c7-be9e-35969e6266f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:37.824335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20419d8c-6609-40c7-be9e-35969e6266f8 service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:37.824646 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20419d8c-6609-40c7-be9e-35969e6266f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:37.825261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20419d8c-6609-40c7-be9e-35969e6266f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:37.834734 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20419d8c-6609-40c7-be9e-35969e6266f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:37.836176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20419d8c-6609-40c7-be9e-35969e6266f8 service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:37.846564 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1420/2849] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:59:37 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:59:38.205257 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a559c55-732d-42cd-81c5-8abdd05aa9a4 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:38.215597 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a559c55-732d-42cd-81c5-8abdd05aa9a4 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:38.219169 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c628a-5288-4808-99ae-5322e23b5606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:38.219771 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c628a-5288-4808-99ae-5322e23b5606 service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:38.220046 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c628a-5288-4808-99ae-5322e23b5606 service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:38.220315 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c628a-5288-4808-99ae-5322e23b5606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:38.253385 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:38.275801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c628a-5288-4808-99ae-5322e23b5606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:38.277980 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c628a-5288-4808-99ae-5322e23b5606 service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:38.287579 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1430/2850] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:59:38 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:59:47.842251 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:48.309069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=11 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:49.595143 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20419d8c-6609-40c7-be9e-35969e6266f8 service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:49.660239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20419d8c-6609-40c7-be9e-35969e6266f8 service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:49.665008 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:49.665641 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:49.665923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:49.666223 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] PATH_INFO: `/v3/users/96656e603e3748e4847c38f4f8761369` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:49.667373 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96656e603e3748e4847c38f4f8761369)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:49.669362 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:49.881592 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96656e603e3748e4847c38f4f8761369'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:49.949143 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96656e603e3748e4847c38f4f8761369'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:49.949900 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96656e603e3748e4847c38f4f8761369'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:49.962868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96656e603e3748e4847c38f4f8761369'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:49.970312 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96656e603e3748e4847c38f4f8761369'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:49.976444 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1421/2851] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:59:49 2022] DELETE /identity/v3/users/96656e603e3748e4847c38f4f8761369 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:49.981155 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b696a4-b033-497f-86c8-e7c418217f73 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:49.981716 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b696a4-b033-497f-86c8-e7c418217f73 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:49.981920 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b696a4-b033-497f-86c8-e7c418217f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:50.001186 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-98b696a4-b033-497f-86c8-e7c418217f73 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:50.002452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-98b696a4-b033-497f-86c8-e7c418217f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:50.062100 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1431/2852] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:50.071715 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:50.073155 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:50.131435 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6789d-1f16-4916-acb4-e3636c865132 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:50.135023 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c29da-743b-4ac8-9ebb-18016349b080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:50.135632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c29da-743b-4ac8-9ebb-18016349b080 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:50.135917 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c29da-743b-4ac8-9ebb-18016349b080 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:50.136194 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c29da-743b-4ac8-9ebb-18016349b080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:50.145811 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c29da-743b-4ac8-9ebb-18016349b080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:50.147286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c29da-743b-4ac8-9ebb-18016349b080 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:50.157693 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1422/2853] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:59:50.622163 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b696a4-b033-497f-86c8-e7c418217f73 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:50.624022 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-98b696a4-b033-497f-86c8-e7c418217f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:50.637499 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b696a4-b033-497f-86c8-e7c418217f73 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:50.642406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:50.643057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:50.643344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:50.643615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] PATH_INFO: `/v3/projects/04290bc917cf4f55969f7b6da83ef264` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:50.644836 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04290bc917cf4f55969f7b6da83ef264)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:50.646784 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:50.781634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04290bc917cf4f55969f7b6da83ef264', 'request_id': 'req-a76bdcf2-5703-43b8-a68b-8d09d87b5223'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:50.917474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04290bc917cf4f55969f7b6da83ef264', 'request_id': 'req-a76bdcf2-5703-43b8-a68b-8d09d87b5223'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:50.923917 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04290bc917cf4f55969f7b6da83ef264', 'request_id': 'req-a76bdcf2-5703-43b8-a68b-8d09d87b5223'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:50.935438 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] The token cache is being invalidate because project 04290bc917cf4f55969f7b6da83ef264 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:59:50.938443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:50.938443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:50.938443 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1432/2854] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:59:50 2022] DELETE /identity/v3/projects/04290bc917cf4f55969f7b6da83ef264 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:50.942107 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c29da-743b-4ac8-9ebb-18016349b080 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:50.952533 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c29da-743b-4ac8-9ebb-18016349b080 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:50.971558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:51.009065 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0648ca-fea9-4854-8e3f-86f201089028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:51.009689 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0648ca-fea9-4854-8e3f-86f201089028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:51.009969 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0648ca-fea9-4854-8e3f-86f201089028 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:51.010285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0648ca-fea9-4854-8e3f-86f201089028 admin admin] PATH_INFO: `/v3/users/2fccedd5c6cd4d949d556009fe8f4d37` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:51.011422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0648ca-fea9-4854-8e3f-86f201089028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fccedd5c6cd4d949d556009fe8f4d37)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:51.013359 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0648ca-fea9-4854-8e3f-86f201089028 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:51.328942 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6e0648ca-fea9-4854-8e3f-86f201089028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fccedd5c6cd4d949d556009fe8f4d37'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:51.437790 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6e0648ca-fea9-4854-8e3f-86f201089028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fccedd5c6cd4d949d556009fe8f4d37'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:51.438455 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6e0648ca-fea9-4854-8e3f-86f201089028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fccedd5c6cd4d949d556009fe8f4d37'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:51.451758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6e0648ca-fea9-4854-8e3f-86f201089028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fccedd5c6cd4d949d556009fe8f4d37'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:51.459321 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6e0648ca-fea9-4854-8e3f-86f201089028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fccedd5c6cd4d949d556009fe8f4d37'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:51.465379 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1423/2855] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:59:50 2022] DELETE /identity/v3/users/2fccedd5c6cd4d949d556009fe8f4d37 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:51.928416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:51.929907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:51.943314 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bdcf2-5703-43b8-a68b-8d09d87b5223 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:52.001772 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:52.002724 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:52.002841 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:52.003044 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] PATH_INFO: `/v3/projects/0a3be3ba1cf6426584870abab32177db` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:52.004198 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a3be3ba1cf6426584870abab32177db)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:52.006136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:52.210426 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:52.226878 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a3be3ba1cf6426584870abab32177db', 'request_id': 'req-f819d540-e016-46a7-8a73-efd5877be8ad'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:52.323121 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a3be3ba1cf6426584870abab32177db', 'request_id': 'req-f819d540-e016-46a7-8a73-efd5877be8ad'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:52.329749 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a3be3ba1cf6426584870abab32177db', 'request_id': 'req-f819d540-e016-46a7-8a73-efd5877be8ad'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:52.340880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] The token cache is being invalidate because project 0a3be3ba1cf6426584870abab32177db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:59:52.341165 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:52.342444 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-f819d540-e016-46a7-8a73-efd5877be8ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:52.343952 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1433/2856] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:59:51 2022] DELETE /identity/v3/projects/0a3be3ba1cf6426584870abab32177db => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:52.349032 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a347b-f3f4-475e-9ee4-84dcdd71f285 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:52.349527 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a347b-f3f4-475e-9ee4-84dcdd71f285 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:52.349644 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a347b-f3f4-475e-9ee4-84dcdd71f285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:52.370586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-242a347b-f3f4-475e-9ee4-84dcdd71f285 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:52.371777 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-242a347b-f3f4-475e-9ee4-84dcdd71f285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:52.432563 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1424/2857] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:52.441345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114731a0-b1bc-43ed-8e8e-00e2dcbbb5cc None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:52.453262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114731a0-b1bc-43ed-8e8e-00e2dcbbb5cc None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:52.453262 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114731a0-b1bc-43ed-8e8e-00e2dcbbb5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:52.462562 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-114731a0-b1bc-43ed-8e8e-00e2dcbbb5cc None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:52.463707 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-114731a0-b1bc-43ed-8e8e-00e2dcbbb5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:52.472293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1434/2858] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:59:52.476598 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a347b-f3f4-475e-9ee4-84dcdd71f285 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:52.486339 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a347b-f3f4-475e-9ee4-84dcdd71f285 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:52.490894 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612dfc8e-6e5c-4a0e-88f7-128c5920c9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:52.491497 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dfc8e-6e5c-4a0e-88f7-128c5920c9ca admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:52.491773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dfc8e-6e5c-4a0e-88f7-128c5920c9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:52.492041 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dfc8e-6e5c-4a0e-88f7-128c5920c9ca admin admin] PATH_INFO: `/v3/domains` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:52.492721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612dfc8e-6e5c-4a0e-88f7-128c5920c9ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91912) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:59:52.493076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612dfc8e-6e5c-4a0e-88f7-128c5920c9ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:52.494672 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612dfc8e-6e5c-4a0e-88f7-128c5920c9ca admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:52.506476 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1425/2859] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 21:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:59:52.510781 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114731a0-b1bc-43ed-8e8e-00e2dcbbb5cc None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:52.520393 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114731a0-b1bc-43ed-8e8e-00e2dcbbb5cc None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:52.525389 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92914a2-fbea-4cb2-9f35-1e13b0e7a59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:52.526023 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92914a2-fbea-4cb2-9f35-1e13b0e7a59a admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:52.526320 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92914a2-fbea-4cb2-9f35-1e13b0e7a59a admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:52.526597 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92914a2-fbea-4cb2-9f35-1e13b0e7a59a admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:52.527274 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92914a2-fbea-4cb2-9f35-1e13b0e7a59a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:52.529835 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92914a2-fbea-4cb2-9f35-1e13b0e7a59a admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:52.640298 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1435/2860] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:59:52 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:59:52.646541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612dfc8e-6e5c-4a0e-88f7-128c5920c9ca admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:52.657292 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612dfc8e-6e5c-4a0e-88f7-128c5920c9ca admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:52.685978 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:52.720726 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eaa42-1732-419d-8249-8dea59bed54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:52.721258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eaa42-1732-419d-8249-8dea59bed54d admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:52.721544 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eaa42-1732-419d-8249-8dea59bed54d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:52.721857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eaa42-1732-419d-8249-8dea59bed54d admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:52.722569 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783eaa42-1732-419d-8249-8dea59bed54d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:52.724548 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783eaa42-1732-419d-8249-8dea59bed54d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:53.140521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1426/2861] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:59:52 2022] POST /identity/v3/users => generated 287 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:59:53.144630 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92914a2-fbea-4cb2-9f35-1e13b0e7a59a admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:53.155497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92914a2-fbea-4cb2-9f35-1e13b0e7a59a admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:53.160182 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e94083-e544-43a2-9b98-8ab17de5e32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:53.160780 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e94083-e544-43a2-9b98-8ab17de5e32b admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:53.161057 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e94083-e544-43a2-9b98-8ab17de5e32b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:53.161392 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e94083-e544-43a2-9b98-8ab17de5e32b admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:53.162078 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e94083-e544-43a2-9b98-8ab17de5e32b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:53.163658 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e94083-e544-43a2-9b98-8ab17de5e32b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:53.175753 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1436/2862] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:59:53.178986 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eaa42-1732-419d-8249-8dea59bed54d admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:53.189829 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eaa42-1732-419d-8249-8dea59bed54d admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:53.194422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b21d2e-f9c1-4ebd-86fb-cc5fa41c07f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:53.195024 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b21d2e-f9c1-4ebd-86fb-cc5fa41c07f5 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:53.195306 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b21d2e-f9c1-4ebd-86fb-cc5fa41c07f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:53.195608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b21d2e-f9c1-4ebd-86fb-cc5fa41c07f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:53.196291 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b21d2e-f9c1-4ebd-86fb-cc5fa41c07f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:53.197877 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b21d2e-f9c1-4ebd-86fb-cc5fa41c07f5 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:53.208784 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1427/2863] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:59:53.212051 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e94083-e544-43a2-9b98-8ab17de5e32b admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:53.222451 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e94083-e544-43a2-9b98-8ab17de5e32b admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:53.227030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42812e-ac75-404d-99fd-951f7ecc9eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:53.227686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42812e-ac75-404d-99fd-951f7ecc9eba admin admin] REQUEST_METHOD: `PUT` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:53.227972 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42812e-ac75-404d-99fd-951f7ecc9eba admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:53.228252 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42812e-ac75-404d-99fd-951f7ecc9eba admin admin] PATH_INFO: `/v3/projects/e52fe4cc5fcd422cbbaaf3fac37e95c2/users/1862162df2284128badfa801a172475f/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:53.243891 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42812e-ac75-404d-99fd-951f7ecc9eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52fe4cc5fcd422cbbaaf3fac37e95c2, user_id=1862162df2284128badfa801a172475f, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:53.246560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42812e-ac75-404d-99fd-951f7ecc9eba admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:53.349384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1437/2864] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:59:53 2022] PUT /identity/v3/projects/e52fe4cc5fcd422cbbaaf3fac37e95c2/users/1862162df2284128badfa801a172475f/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:53.414450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fcedf-f5c9-454e-a91c-dfe415956f19 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:53.415001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fcedf-f5c9-454e-a91c-dfe415956f19 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:53.415001 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fcedf-f5c9-454e-a91c-dfe415956f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:53.443269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-951fcedf-f5c9-454e-a91c-dfe415956f19 None None] MFA Rules not processed for user `1862162df2284128badfa801a172475f`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:53.444914 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-951fcedf-f5c9-454e-a91c-dfe415956f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:53.508600 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1428/2865] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:53.513219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42812e-ac75-404d-99fd-951f7ecc9eba admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:53.576679 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42812e-ac75-404d-99fd-951f7ecc9eba admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:53.581489 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fa685-efa0-4da8-a334-07dde651be06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:53.582248 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fa685-efa0-4da8-a334-07dde651be06 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:53.582420 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fa685-efa0-4da8-a334-07dde651be06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:53.582686 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fa685-efa0-4da8-a334-07dde651be06 admin admin] PATH_INFO: `/v3/projects` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:53.583352 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fa685-efa0-4da8-a334-07dde651be06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:53.585224 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fa685-efa0-4da8-a334-07dde651be06 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:53.699158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1438/2866] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 21:59:53 2022] POST /identity/v3/projects => generated 375 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:59:53.703433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fcedf-f5c9-454e-a91c-dfe415956f19 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:53.714201 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fcedf-f5c9-454e-a91c-dfe415956f19 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:53.773377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dbc80-c88b-4570-9789-bd11154f6d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:53.774212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dbc80-c88b-4570-9789-bd11154f6d64 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:53.774500 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dbc80-c88b-4570-9789-bd11154f6d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:53.774780 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dbc80-c88b-4570-9789-bd11154f6d64 admin admin] PATH_INFO: `/v3/users` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:53.775460 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502dbc80-c88b-4570-9789-bd11154f6d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:53.777575 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502dbc80-c88b-4570-9789-bd11154f6d64 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:53.875229 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1429/2867] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 21:59:53 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:59:53.879299 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fa685-efa0-4da8-a334-07dde651be06 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:53.890384 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fa685-efa0-4da8-a334-07dde651be06 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:53.895026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36583321-06e1-4d4f-9dca-3ecf5cfb2562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:53.895631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36583321-06e1-4d4f-9dca-3ecf5cfb2562 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:53.895913 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36583321-06e1-4d4f-9dca-3ecf5cfb2562 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:53.896188 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36583321-06e1-4d4f-9dca-3ecf5cfb2562 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:53.896874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36583321-06e1-4d4f-9dca-3ecf5cfb2562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:53.898497 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36583321-06e1-4d4f-9dca-3ecf5cfb2562 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:53.909757 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1439/2868] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:59:53.913992 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dbc80-c88b-4570-9789-bd11154f6d64 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:53.924266 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dbc80-c88b-4570-9789-bd11154f6d64 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:53.928864 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdd2db-2486-43d6-8d33-5f277648248b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:53.929549 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdd2db-2486-43d6-8d33-5f277648248b admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:53.929793 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdd2db-2486-43d6-8d33-5f277648248b admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:53.930170 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdd2db-2486-43d6-8d33-5f277648248b admin admin] PATH_INFO: `/v3/projects/9b5897710cd341bbb0c0685ecb23dc8f/users/19c658f63c394fa69c886a8b5825a0fb/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:53.946895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdd2db-2486-43d6-8d33-5f277648248b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5897710cd341bbb0c0685ecb23dc8f, user_id=19c658f63c394fa69c886a8b5825a0fb, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:53.949662 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdd2db-2486-43d6-8d33-5f277648248b admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:54.069428 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1430/2869] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:59:53 2022] PUT /identity/v3/projects/9b5897710cd341bbb0c0685ecb23dc8f/users/19c658f63c394fa69c886a8b5825a0fb/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:54.073487 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36583321-06e1-4d4f-9dca-3ecf5cfb2562 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:54.084410 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36583321-06e1-4d4f-9dca-3ecf5cfb2562 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:54.104974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:54.142650 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e967e-660b-4440-bf0d-040cd32cf2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:54.143270 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e967e-660b-4440-bf0d-040cd32cf2ca admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:54.143553 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e967e-660b-4440-bf0d-040cd32cf2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:54.143830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e967e-660b-4440-bf0d-040cd32cf2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:54.144523 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e967e-660b-4440-bf0d-040cd32cf2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:54.146258 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e967e-660b-4440-bf0d-040cd32cf2ca admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:54.156137 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1440/2870] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 21:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:59:54.160125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdd2db-2486-43d6-8d33-5f277648248b admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:54.165502 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:54.171129 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdd2db-2486-43d6-8d33-5f277648248b admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:54.177674 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfd884-ee63-488b-ba05-72d1310ca80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:54.181682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfd884-ee63-488b-ba05-72d1310ca80c admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:54.181682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfd884-ee63-488b-ba05-72d1310ca80c admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:54.181682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfd884-ee63-488b-ba05-72d1310ca80c admin admin] PATH_INFO: `/v3/projects/9b5897710cd341bbb0c0685ecb23dc8f/users/19c658f63c394fa69c886a8b5825a0fb/roles/b39c7fb6d2e34182bdda29361f6c005e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:54.181682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfd884-ee63-488b-ba05-72d1310ca80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5897710cd341bbb0c0685ecb23dc8f, user_id=19c658f63c394fa69c886a8b5825a0fb, role_id=b39c7fb6d2e34182bdda29361f6c005e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:54.183503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfd884-ee63-488b-ba05-72d1310ca80c admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:54.194405 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1431/2871] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 21:59:54 2022] PUT /identity/v3/projects/9b5897710cd341bbb0c0685ecb23dc8f/users/19c658f63c394fa69c886a8b5825a0fb/roles/b39c7fb6d2e34182bdda29361f6c005e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:54.205185 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325246b-1b2b-4e78-98ed-acc79aae6189 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:54.205397 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325246b-1b2b-4e78-98ed-acc79aae6189 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:54.205673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325246b-1b2b-4e78-98ed-acc79aae6189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:54.233605 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-2325246b-1b2b-4e78-98ed-acc79aae6189 None None] MFA Rules not processed for user `19c658f63c394fa69c886a8b5825a0fb`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:54.235040 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2325246b-1b2b-4e78-98ed-acc79aae6189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:54.303133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1441/2872] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:54.314789 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfd884-ee63-488b-ba05-72d1310ca80c admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:54.316462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ebbfd884-ee63-488b-ba05-72d1310ca80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:54.372967 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfd884-ee63-488b-ba05-72d1310ca80c admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:54.377156 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd1698-8134-48a1-98d0-7b568f158fee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:54.377761 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1698-8134-48a1-98d0-7b568f158fee service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:54.378067 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1698-8134-48a1-98d0-7b568f158fee service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:54.378341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1698-8134-48a1-98d0-7b568f158fee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:54.436241 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd1698-8134-48a1-98d0-7b568f158fee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:54.438388 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd1698-8134-48a1-98d0-7b568f158fee service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:54.447466 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1432/2873] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:59:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:59:58.648709 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325246b-1b2b-4e78-98ed-acc79aae6189 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:58.712402 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325246b-1b2b-4e78-98ed-acc79aae6189 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:58.716887 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:58.717515 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:58.717792 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:58.718092 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] PATH_INFO: `/v3/users/1862162df2284128badfa801a172475f` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:58.719368 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1862162df2284128badfa801a172475f)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:58.721414 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:58.994504 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1862162df2284128badfa801a172475f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:58.996242 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:59.168594 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1862162df2284128badfa801a172475f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:59.181642 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1862162df2284128badfa801a172475f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:59.182317 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1862162df2284128badfa801a172475f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:59.190634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1862162df2284128badfa801a172475f'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:59.196591 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1442/2874] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 21:59:58 2022] DELETE /identity/v3/users/1862162df2284128badfa801a172475f => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:59:59.201190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7681b-b1cf-4fc1-80aa-8fa223893d57 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:59.201809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7681b-b1cf-4fc1-80aa-8fa223893d57 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:59.201809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7681b-b1cf-4fc1-80aa-8fa223893d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:59.220519 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-1ae7681b-b1cf-4fc1-80aa-8fa223893d57 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:59:59.221752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1ae7681b-b1cf-4fc1-80aa-8fa223893d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:59.281722 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1433/2875] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 21:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:59:59.291998 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:59.293474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:59.350575 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d236d-86f8-4359-a210-60ee9b071445 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:59.354264 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037c906-99ad-45b9-a2c0-99fd66857286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:59.354886 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037c906-99ad-45b9-a2c0-99fd66857286 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:59.355172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037c906-99ad-45b9-a2c0-99fd66857286 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:59.355445 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037c906-99ad-45b9-a2c0-99fd66857286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:59.365471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037c906-99ad-45b9-a2c0-99fd66857286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:59.366918 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037c906-99ad-45b9-a2c0-99fd66857286 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:59.376334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1443/2876] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 21:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:59:59.827557 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7681b-b1cf-4fc1-80aa-8fa223893d57 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:59:59.829090 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1ae7681b-b1cf-4fc1-80aa-8fa223893d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:59:59.841891 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7681b-b1cf-4fc1-80aa-8fa223893d57 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:59:59.848619 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:59:59.849278 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:59:59.849566 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:59:59.849855 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] PATH_INFO: `/v3/projects/e52fe4cc5fcd422cbbaaf3fac37e95c2` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:59:59.851087 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e52fe4cc5fcd422cbbaaf3fac37e95c2)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:59:59.853212 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:59:59.968574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:59:59.975529 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e52fe4cc5fcd422cbbaaf3fac37e95c2', 'request_id': 'req-1538e385-779f-4056-84d4-a4f830042beb'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:59:59.981765 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e52fe4cc5fcd422cbbaaf3fac37e95c2', 'request_id': 'req-1538e385-779f-4056-84d4-a4f830042beb'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:00.066261 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e52fe4cc5fcd422cbbaaf3fac37e95c2', 'request_id': 'req-1538e385-779f-4056-84d4-a4f830042beb'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:00.077006 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] The token cache is being invalidate because project e52fe4cc5fcd422cbbaaf3fac37e95c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:00:00.077236 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:00.077857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:00.079433 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1434/2877] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 21:59:59 2022] DELETE /identity/v3/projects/e52fe4cc5fcd422cbbaaf3fac37e95c2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:00:00.083660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037c906-99ad-45b9-a2c0-99fd66857286 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:00.093928 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037c906-99ad-45b9-a2c0-99fd66857286 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:00.151802 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30858f-8c38-4b7a-8f89-cc157d5676ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:00.152437 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30858f-8c38-4b7a-8f89-cc157d5676ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:00.152718 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30858f-8c38-4b7a-8f89-cc157d5676ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:00.153008 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30858f-8c38-4b7a-8f89-cc157d5676ce admin admin] PATH_INFO: `/v3/users/19c658f63c394fa69c886a8b5825a0fb` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:00.154303 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30858f-8c38-4b7a-8f89-cc157d5676ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c658f63c394fa69c886a8b5825a0fb)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:00.156200 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30858f-8c38-4b7a-8f89-cc157d5676ce admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:00.359141 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2e30858f-8c38-4b7a-8f89-cc157d5676ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c658f63c394fa69c886a8b5825a0fb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:00.457653 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2e30858f-8c38-4b7a-8f89-cc157d5676ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c658f63c394fa69c886a8b5825a0fb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:00.469130 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:00:00.470537 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2e30858f-8c38-4b7a-8f89-cc157d5676ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c658f63c394fa69c886a8b5825a0fb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:00.471218 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2e30858f-8c38-4b7a-8f89-cc157d5676ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c658f63c394fa69c886a8b5825a0fb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:00.478493 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-2e30858f-8c38-4b7a-8f89-cc157d5676ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c658f63c394fa69c886a8b5825a0fb'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:00.484205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1444/2878] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 22:00:00 2022] DELETE /identity/v3/users/19c658f63c394fa69c886a8b5825a0fb => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:00:00.979977 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:00.981787 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:00.996124 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538e385-779f-4056-84d4-a4f830042beb admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:01.057972 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:01.058716 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:01.058998 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:01.059272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] PATH_INFO: `/v3/projects/9b5897710cd341bbb0c0685ecb23dc8f` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:01.060436 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b5897710cd341bbb0c0685ecb23dc8f)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:01.062411 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:01.207742 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b5897710cd341bbb0c0685ecb23dc8f', 'request_id': 'req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:01.214104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b5897710cd341bbb0c0685ecb23dc8f', 'request_id': 'req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:01.291558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b5897710cd341bbb0c0685ecb23dc8f', 'request_id': 'req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:01.303968 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] The token cache is being invalidate because project 9b5897710cd341bbb0c0685ecb23dc8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:00:01.304328 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:01.304910 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-6fb51a99-23bd-4d4b-9db9-730c1c32e7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:01.306454 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1435/2879] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 22:00:00 2022] DELETE /identity/v3/projects/9b5897710cd341bbb0c0685ecb23dc8f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:00:01.312403 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a688e-1dc1-43ed-9e34-e59671462eac None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:01.312907 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a688e-1dc1-43ed-9e34-e59671462eac None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:01.313030 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a688e-1dc1-43ed-9e34-e59671462eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:01.333281 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-f63a688e-1dc1-43ed-9e34-e59671462eac None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:00:01.334535 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-f63a688e-1dc1-43ed-9e34-e59671462eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:01.397249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1445/2880] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:00:01.406344 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a44ea-c4fe-40f8-bb6c-f6fe964df7e8 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:01.406629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a44ea-c4fe-40f8-bb6c-f6fe964df7e8 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:01.406955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a44ea-c4fe-40f8-bb6c-f6fe964df7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:01.429385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-6d7a44ea-c4fe-40f8-bb6c-f6fe964df7e8 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:00:01.430630 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6d7a44ea-c4fe-40f8-bb6c-f6fe964df7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:01.438588 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1436/2881] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:00:01.443667 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a688e-1dc1-43ed-9e34-e59671462eac None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:01.453721 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a688e-1dc1-43ed-9e34-e59671462eac None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:01.458431 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d0cd6-880e-4998-9349-179074bd4a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:01.459054 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d0cd6-880e-4998-9349-179074bd4a61 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:01.459334 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d0cd6-880e-4998-9349-179074bd4a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:01.459615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d0cd6-880e-4998-9349-179074bd4a61 admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:01.460344 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d0cd6-880e-4998-9349-179074bd4a61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:00:01.460645 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d0cd6-880e-4998-9349-179074bd4a61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:01.462266 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d0cd6-880e-4998-9349-179074bd4a61 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:01.474874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1446/2882] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 22:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:00:01.479126 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a44ea-c4fe-40f8-bb6c-f6fe964df7e8 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:01.489245 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a44ea-c4fe-40f8-bb6c-f6fe964df7e8 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:01.493857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92eeae-3505-4d5a-b6bb-df98de910722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:01.494503 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92eeae-3505-4d5a-b6bb-df98de910722 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:01.494858 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92eeae-3505-4d5a-b6bb-df98de910722 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:01.495136 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92eeae-3505-4d5a-b6bb-df98de910722 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:01.495814 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92eeae-3505-4d5a-b6bb-df98de910722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:01.497974 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92eeae-3505-4d5a-b6bb-df98de910722 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:01.645402 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1437/2883] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 22:00:01 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:00:01.649827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d0cd6-880e-4998-9349-179074bd4a61 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:01.661231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d0cd6-880e-4998-9349-179074bd4a61 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:01.716488 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a796244-3dc1-4c83-8d13-0dfbeaa3f4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:01.717136 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a796244-3dc1-4c83-8d13-0dfbeaa3f4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:01.717415 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a796244-3dc1-4c83-8d13-0dfbeaa3f4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:01.717687 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a796244-3dc1-4c83-8d13-0dfbeaa3f4a8 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:01.718373 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a796244-3dc1-4c83-8d13-0dfbeaa3f4a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:01.720265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a796244-3dc1-4c83-8d13-0dfbeaa3f4a8 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:01.807294 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1447/2884] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 22:00:01 2022] POST /identity/v3/users => generated 284 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:00:01.811492 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92eeae-3505-4d5a-b6bb-df98de910722 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:01.822333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92eeae-3505-4d5a-b6bb-df98de910722 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:01.827009 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025becae-5091-4178-989f-81462500d6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:01.829649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025becae-5091-4178-989f-81462500d6ab admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:01.829649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025becae-5091-4178-989f-81462500d6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:01.829649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025becae-5091-4178-989f-81462500d6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:01.829649 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025becae-5091-4178-989f-81462500d6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:01.830410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025becae-5091-4178-989f-81462500d6ab admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:01.840450 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1438/2885] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 22:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:00:01.844833 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a796244-3dc1-4c83-8d13-0dfbeaa3f4a8 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:01.855478 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a796244-3dc1-4c83-8d13-0dfbeaa3f4a8 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:01.860469 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e80f5-bc91-4055-b410-f334e5bbc441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:01.861132 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e80f5-bc91-4055-b410-f334e5bbc441 admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:01.861456 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e80f5-bc91-4055-b410-f334e5bbc441 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:01.861742 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e80f5-bc91-4055-b410-f334e5bbc441 admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:01.863496 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e80f5-bc91-4055-b410-f334e5bbc441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:01.865099 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e80f5-bc91-4055-b410-f334e5bbc441 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:01.876710 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1448/2886] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 22:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:00:01.880987 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025becae-5091-4178-989f-81462500d6ab admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:01.891475 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025becae-5091-4178-989f-81462500d6ab admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:01.896869 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad86987-b692-45b8-9059-6a09cf6c1eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:01.897565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad86987-b692-45b8-9059-6a09cf6c1eb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:01.897850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad86987-b692-45b8-9059-6a09cf6c1eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:01.898165 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad86987-b692-45b8-9059-6a09cf6c1eb4 admin admin] PATH_INFO: `/v3/projects/8bcd0a55a2d449c2a3f5db6484884c92/users/b0ed3208d7344bd892b755acd81c106c/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:01.913467 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad86987-b692-45b8-9059-6a09cf6c1eb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bcd0a55a2d449c2a3f5db6484884c92, user_id=b0ed3208d7344bd892b755acd81c106c, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:01.916142 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad86987-b692-45b8-9059-6a09cf6c1eb4 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:01.918385 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:00:02.008835 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1439/2887] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 22:00:01 2022] PUT /identity/v3/projects/8bcd0a55a2d449c2a3f5db6484884c92/users/b0ed3208d7344bd892b755acd81c106c/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:00:02.021119 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fef3f-266a-419b-bcf4-f71573ccda6f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:02.021519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fef3f-266a-419b-bcf4-f71573ccda6f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:02.021829 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fef3f-266a-419b-bcf4-f71573ccda6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:02.050346 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-394fef3f-266a-419b-bcf4-f71573ccda6f None None] MFA Rules not processed for user `b0ed3208d7344bd892b755acd81c106c`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:00:02.051452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-394fef3f-266a-419b-bcf4-f71573ccda6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:02.093143 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:00:02.116171 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1449/2888] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:00:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:00:02.128341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad86987-b692-45b8-9059-6a09cf6c1eb4 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:02.130014 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-6ad86987-b692-45b8-9059-6a09cf6c1eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:02.210347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad86987-b692-45b8-9059-6a09cf6c1eb4 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:02.214186 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348f07a-44ec-405c-9b10-bee0e97be87d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a9a6bcfa1b469298774e09a8912b93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:02.214798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348f07a-44ec-405c-9b10-bee0e97be87d service glance] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:02.215076 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348f07a-44ec-405c-9b10-bee0e97be87d service glance] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:02.215345 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348f07a-44ec-405c-9b10-bee0e97be87d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:02.224429 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348f07a-44ec-405c-9b10-bee0e97be87d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:02.225850 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348f07a-44ec-405c-9b10-bee0e97be87d service glance] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:02.234559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1440/2889] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:00:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:00:12.180128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=8 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:00:12.230771 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:INSERT=1,keystone:SELECT=37 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:00:48.927819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fef3f-266a-419b-bcf4-f71573ccda6f None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:48.989196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fef3f-266a-419b-bcf4-f71573ccda6f None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:48.993791 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:48.994464 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:48.994744 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:48.995017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] PATH_INFO: `/v3/users/b0ed3208d7344bd892b755acd81c106c` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:48.996184 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0ed3208d7344bd892b755acd81c106c)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:48.998416 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:49.185231 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0ed3208d7344bd892b755acd81c106c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:49.259026 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0ed3208d7344bd892b755acd81c106c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:49.272731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0ed3208d7344bd892b755acd81c106c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:49.273460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0ed3208d7344bd892b755acd81c106c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:49.281055 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0ed3208d7344bd892b755acd81c106c'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:49.286948 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1450/2890] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 22:00:48 2022] DELETE /identity/v3/users/b0ed3208d7344bd892b755acd81c106c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:00:49.296432 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476b1ee-8402-42df-8e2a-371365a07a2d None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:49.296962 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476b1ee-8402-42df-8e2a-371365a07a2d None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:49.297071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476b1ee-8402-42df-8e2a-371365a07a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:49.316660 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-c476b1ee-8402-42df-8e2a-371365a07a2d None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:00:49.317875 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c476b1ee-8402-42df-8e2a-371365a07a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:49.377273 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1441/2891] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:00:49.387801 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:49.389230 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:49.447610 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149724f-8c0d-44ae-a51b-a09f35086a47 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:49.452058 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c7389-5769-4b94-9904-b5393b34fbe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:49.452673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c7389-5769-4b94-9904-b5393b34fbe5 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:49.452950 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c7389-5769-4b94-9904-b5393b34fbe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:49.453219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c7389-5769-4b94-9904-b5393b34fbe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:49.462613 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c7389-5769-4b94-9904-b5393b34fbe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:49.464037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c7389-5769-4b94-9904-b5393b34fbe5 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:49.474880 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1451/2892] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:00:49.940848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476b1ee-8402-42df-8e2a-371365a07a2d None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:49.942606 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c476b1ee-8402-42df-8e2a-371365a07a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:49.955221 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476b1ee-8402-42df-8e2a-371365a07a2d None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:49.960880 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:49.962337 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:49.962804 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:49.963092 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] PATH_INFO: `/v3/projects/8bcd0a55a2d449c2a3f5db6484884c92` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:49.964412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bcd0a55a2d449c2a3f5db6484884c92)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:49.967422 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:50.079853 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bcd0a55a2d449c2a3f5db6484884c92', 'request_id': 'req-65ad2b16-2800-4b88-b9f7-b553d9e8df18'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:50.086074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bcd0a55a2d449c2a3f5db6484884c92', 'request_id': 'req-65ad2b16-2800-4b88-b9f7-b553d9e8df18'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:50.170552 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bcd0a55a2d449c2a3f5db6484884c92', 'request_id': 'req-65ad2b16-2800-4b88-b9f7-b553d9e8df18'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:50.181559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] The token cache is being invalidate because project 8bcd0a55a2d449c2a3f5db6484884c92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:00:50.181848 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:50.182510 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-65ad2b16-2800-4b88-b9f7-b553d9e8df18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:00:50.184094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1442/2893] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 22:00:49 2022] DELETE /identity/v3/projects/8bcd0a55a2d449c2a3f5db6484884c92 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:00:50.189147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ee733-78e0-434f-8204-2c98827e0cf6 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:50.189660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ee733-78e0-434f-8204-2c98827e0cf6 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:50.189773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ee733-78e0-434f-8204-2c98827e0cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:50.209816 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-429ee733-78e0-434f-8204-2c98827e0cf6 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:00:50.211062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-429ee733-78e0-434f-8204-2c98827e0cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:50.229150 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:00:50.273110 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1452/2894] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:00:50.283099 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebcbd9-ca0e-4d7d-b3f3-878ca3b0b7d8 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:50.283413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebcbd9-ca0e-4d7d-b3f3-878ca3b0b7d8 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:50.283737 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebcbd9-ca0e-4d7d-b3f3-878ca3b0b7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:50.304456 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-71ebcbd9-ca0e-4d7d-b3f3-878ca3b0b7d8 None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:00:50.305735 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-71ebcbd9-ca0e-4d7d-b3f3-878ca3b0b7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:50.314285 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1443/2895] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:00:50.318603 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ee733-78e0-434f-8204-2c98827e0cf6 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:50.328338 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ee733-78e0-434f-8204-2c98827e0cf6 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:50.332899 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d5e77-d259-4d44-957a-784beeede29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:50.333519 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d5e77-d259-4d44-957a-784beeede29e admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:50.333797 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d5e77-d259-4d44-957a-784beeede29e admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:50.334098 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d5e77-d259-4d44-957a-784beeede29e admin admin] PATH_INFO: `/v3/domains` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:50.334794 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d5e77-d259-4d44-957a-784beeede29e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91913) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:00:50.335097 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d5e77-d259-4d44-957a-784beeede29e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:50.336696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d5e77-d259-4d44-957a-784beeede29e admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:50.349474 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1453/2896] 10.222.0.73 () {58 vars in 1153 bytes} [Thu Aug 4 22:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:00:50.353961 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebcbd9-ca0e-4d7d-b3f3-878ca3b0b7d8 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:50.364623 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebcbd9-ca0e-4d7d-b3f3-878ca3b0b7d8 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:50.369342 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8dfe4-c5e3-429a-bd27-b23da07a4e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:50.369955 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8dfe4-c5e3-429a-bd27-b23da07a4e92 admin admin] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:50.370274 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8dfe4-c5e3-429a-bd27-b23da07a4e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:50.370550 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8dfe4-c5e3-429a-bd27-b23da07a4e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:50.371335 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8dfe4-c5e3-429a-bd27-b23da07a4e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:50.373608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8dfe4-c5e3-429a-bd27-b23da07a4e92 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:50.472654 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1444/2897] 10.222.0.73 () {60 vars in 1153 bytes} [Thu Aug 4 22:00:50 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:00:50.477070 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d5e77-d259-4d44-957a-784beeede29e admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:50.487659 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d5e77-d259-4d44-957a-784beeede29e admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:50.545698 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28944a1-be4c-4dd3-9dbf-a81cad964801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:50.546361 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28944a1-be4c-4dd3-9dbf-a81cad964801 admin admin] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:50.546642 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28944a1-be4c-4dd3-9dbf-a81cad964801 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:50.546913 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28944a1-be4c-4dd3-9dbf-a81cad964801 admin admin] PATH_INFO: `/v3/users` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:50.547582 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28944a1-be4c-4dd3-9dbf-a81cad964801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:50.549482 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28944a1-be4c-4dd3-9dbf-a81cad964801 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:50.619219 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1454/2898] 10.222.0.73 () {60 vars in 1144 bytes} [Thu Aug 4 22:00:50 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:00:50.623895 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8dfe4-c5e3-429a-bd27-b23da07a4e92 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:50.634521 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8dfe4-c5e3-429a-bd27-b23da07a4e92 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:50.640089 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffdb28-8c19-45dd-add4-ca443fe16096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:50.640702 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffdb28-8c19-45dd-add4-ca443fe16096 admin admin] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:50.640989 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffdb28-8c19-45dd-add4-ca443fe16096 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:50.641294 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffdb28-8c19-45dd-add4-ca443fe16096 admin admin] PATH_INFO: `/v3/roles` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:50.641960 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfffdb28-8c19-45dd-add4-ca443fe16096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:50.643586 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfffdb28-8c19-45dd-add4-ca443fe16096 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:50.654791 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1445/2899] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 22:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:00:50.659062 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28944a1-be4c-4dd3-9dbf-a81cad964801 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:50.669725 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28944a1-be4c-4dd3-9dbf-a81cad964801 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:50.678852 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755a8f6-03c4-4090-aa5a-ff2b1dfe598b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:50.679471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a8f6-03c4-4090-aa5a-ff2b1dfe598b admin admin] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:50.679750 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a8f6-03c4-4090-aa5a-ff2b1dfe598b admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:50.680017 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a8f6-03c4-4090-aa5a-ff2b1dfe598b admin admin] PATH_INFO: `/v3/roles` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:50.680827 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755a8f6-03c4-4090-aa5a-ff2b1dfe598b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:50.682452 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755a8f6-03c4-4090-aa5a-ff2b1dfe598b admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:50.693988 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1455/2900] 10.222.0.73 () {58 vars in 1122 bytes} [Thu Aug 4 22:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:00:50.698384 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffdb28-8c19-45dd-add4-ca443fe16096 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:50.709256 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffdb28-8c19-45dd-add4-ca443fe16096 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:50.713868 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9ec9b-0a0e-4237-b430-2bc2d412f604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:50.714561 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9ec9b-0a0e-4237-b430-2bc2d412f604 admin admin] REQUEST_METHOD: `PUT` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:50.715508 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9ec9b-0a0e-4237-b430-2bc2d412f604 admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:50.716016 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9ec9b-0a0e-4237-b430-2bc2d412f604 admin admin] PATH_INFO: `/v3/projects/0e75154a4e9e4562b0757fc22af26d1e/users/037873250f324a82ba88d4ee151bfa2e/roles/36291c067d1949cfae0d704585f036c8` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:50.731216 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9ec9b-0a0e-4237-b430-2bc2d412f604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e75154a4e9e4562b0757fc22af26d1e, user_id=037873250f324a82ba88d4ee151bfa2e, role_id=36291c067d1949cfae0d704585f036c8)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:50.733879 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9ec9b-0a0e-4237-b430-2bc2d412f604 admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:50.812307 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1446/2901] 10.222.0.73 () {60 vars in 1483 bytes} [Thu Aug 4 22:00:50 2022] PUT /identity/v3/projects/0e75154a4e9e4562b0757fc22af26d1e/users/037873250f324a82ba88d4ee151bfa2e/roles/36291c067d1949cfae0d704585f036c8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:00:50.817433 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764160f4-cc57-4ba1-a3c9-b92c90a7180a None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:50.817840 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764160f4-cc57-4ba1-a3c9-b92c90a7180a None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:50.818148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764160f4-cc57-4ba1-a3c9-b92c90a7180a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:50.837889 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-764160f4-cc57-4ba1-a3c9-b92c90a7180a None None] MFA Rules not processed for user `ec8458e10b6c4d69ba0ad246e923be31`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:00:50.839128 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-764160f4-cc57-4ba1-a3c9-b92c90a7180a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:50.900634 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1456/2902] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:00:50.910792 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9ec9b-0a0e-4237-b430-2bc2d412f604 admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:50.912495 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-72e9ec9b-0a0e-4237-b430-2bc2d412f604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:50.925752 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:00:50.970177 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9ec9b-0a0e-4237-b430-2bc2d412f604 admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:50.973782 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf2170-98e6-4c33-8a01-8308c4cca851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:50.974412 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf2170-98e6-4c33-8a01-8308c4cca851 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:50.978188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf2170-98e6-4c33-8a01-8308c4cca851 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:50.978188 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf2170-98e6-4c33-8a01-8308c4cca851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:50.984157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf2170-98e6-4c33-8a01-8308c4cca851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:50.985587 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf2170-98e6-4c33-8a01-8308c4cca851 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:50.995054 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1447/2903] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:00:52.249069 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c0d9a-8fc6-4151-ba20-e2374c8f216f None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:52.249624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c0d9a-8fc6-4151-ba20-e2374c8f216f None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:52.249624 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c0d9a-8fc6-4151-ba20-e2374c8f216f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:52.280158 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-147c0d9a-8fc6-4151-ba20-e2374c8f216f None None] MFA Rules not processed for user `037873250f324a82ba88d4ee151bfa2e`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:00:52.281391 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-147c0d9a-8fc6-4151-ba20-e2374c8f216f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:52.300088 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:00:52.356479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1457/2904] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:00:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:00:52.368310 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf2170-98e6-4c33-8a01-8308c4cca851 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:52.369629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-c5bf2170-98e6-4c33-8a01-8308c4cca851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:52.433020 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf2170-98e6-4c33-8a01-8308c4cca851 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:52.437321 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08a9de-288c-4e79-833f-5d9ebb242c2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:52.437963 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a9de-288c-4e79-833f-5d9ebb242c2f service cinder] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:52.438269 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a9de-288c-4e79-833f-5d9ebb242c2f service cinder] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:52.438541 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a9de-288c-4e79-833f-5d9ebb242c2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:52.453419 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08a9de-288c-4e79-833f-5d9ebb242c2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:52.454898 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08a9de-288c-4e79-833f-5d9ebb242c2f service cinder] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:52.463664 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1448/2905] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:00:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:00:54.223858 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f68e-497f-4d4f-9310-255b71783688 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:54.224439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f68e-497f-4d4f-9310-255b71783688 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:54.224439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f68e-497f-4d4f-9310-255b71783688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:54.244037 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-a364f68e-497f-4d4f-9310-255b71783688 None None] MFA Rules not processed for user `037873250f324a82ba88d4ee151bfa2e`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:00:54.246832 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a364f68e-497f-4d4f-9310-255b71783688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:54.253656 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1458/2906] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:00:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:00:54.293713 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08a9de-288c-4e79-833f-5d9ebb242c2f service cinder] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:54.295220 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-ca08a9de-288c-4e79-833f-5d9ebb242c2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:54.354219 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08a9de-288c-4e79-833f-5d9ebb242c2f service cinder] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:54.358694 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228591c-5d91-4ef3-bcce-12c5f45cd288 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a9a6bcfa1b469298774e09a8912b93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:54.359347 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228591c-5d91-4ef3-bcce-12c5f45cd288 service glance] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:54.359629 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228591c-5d91-4ef3-bcce-12c5f45cd288 service glance] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:54.359923 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228591c-5d91-4ef3-bcce-12c5f45cd288 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:54.361824 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1228591c-5d91-4ef3-bcce-12c5f45cd288 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:54.440911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=100 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:00:54.458818 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228591c-5d91-4ef3-bcce-12c5f45cd288 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:54.468333 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228591c-5d91-4ef3-bcce-12c5f45cd288 service glance] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:54.483081 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1449/2907] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:00:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:00:54.700042 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364f68e-497f-4d4f-9310-255b71783688 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:54.710888 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364f68e-497f-4d4f-9310-255b71783688 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:54.714471 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6133d0-d872-49e6-b10e-0cbedb2e4686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:54.716009 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6133d0-d872-49e6-b10e-0cbedb2e4686 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:54.716388 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6133d0-d872-49e6-b10e-0cbedb2e4686 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:54.716663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6133d0-d872-49e6-b10e-0cbedb2e4686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:54.718440 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bf6133d0-d872-49e6-b10e-0cbedb2e4686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:54.729318 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6133d0-d872-49e6-b10e-0cbedb2e4686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:54.731620 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6133d0-d872-49e6-b10e-0cbedb2e4686 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:54.740754 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1459/2908] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:00:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:00:55.223504 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228591c-5d91-4ef3-bcce-12c5f45cd288 service glance] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:00:55.225004 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-1228591c-5d91-4ef3-bcce-12c5f45cd288 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:55.237377 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228591c-5d91-4ef3-bcce-12c5f45cd288 service glance] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:00:55.241666 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730d561-f6bf-4900-b521-df6bdf096c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:00:55.241666 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730d561-f6bf-4900-b521-df6bdf096c62 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:00:55.242125 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730d561-f6bf-4900-b521-df6bdf096c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:00:55.242408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730d561-f6bf-4900-b521-df6bdf096c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:00:55.244176 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3730d561-f6bf-4900-b521-df6bdf096c62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:00:55.259626 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730d561-f6bf-4900-b521-df6bdf096c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:00:55.261075 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730d561-f6bf-4900-b521-df6bdf096c62 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:00:55.273916 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1450/2909] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:00:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:01:04.736473 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=28 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:01:05.269038 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=16 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:01:36.974333 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32d50c-bf96-45f7-af86-1eaab32e13ac None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:01:36.976254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32d50c-bf96-45f7-af86-1eaab32e13ac None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:01:36.976254 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32d50c-bf96-45f7-af86-1eaab32e13ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:01:36.993623 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-7d32d50c-bf96-45f7-af86-1eaab32e13ac None None] MFA Rules not processed for user `292eee6c79214436962643db0a840708`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:01:36.994800 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-7d32d50c-bf96-45f7-af86-1eaab32e13ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:01:37.049673 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1460/2910] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:01:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:01:47.043032 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=25 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:02:07.745146 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730d561-f6bf-4900-b521-df6bdf096c62 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:02:07.746857 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-3730d561-f6bf-4900-b521-df6bdf096c62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:02:07.760911 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730d561-f6bf-4900-b521-df6bdf096c62 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:02:07.766798 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6371a-7fd6-44db-b3e7-209ce84b728d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:02:07.767436 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6371a-7fd6-44db-b3e7-209ce84b728d service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:02:07.767720 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6371a-7fd6-44db-b3e7-209ce84b728d service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:02:07.768017 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6371a-7fd6-44db-b3e7-209ce84b728d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:02:07.769769 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-e2d6371a-7fd6-44db-b3e7-209ce84b728d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:02:07.789327 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6371a-7fd6-44db-b3e7-209ce84b728d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:02:07.790990 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6371a-7fd6-44db-b3e7-209ce84b728d service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:02:07.802998 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1451/2911] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:02:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:02:17.797991 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=6 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:02:20.216527 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990392b1-95ce-4baa-9361-4ae4902cb493 None None] REQUEST_METHOD: `POST` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:02:20.217077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990392b1-95ce-4baa-9361-4ae4902cb493 None None] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:02:20.217077 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990392b1-95ce-4baa-9361-4ae4902cb493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:02:20.237542 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.auth.core [None req-990392b1-95ce-4baa-9361-4ae4902cb493 None None] MFA Rules not processed for user `037873250f324a82ba88d4ee151bfa2e`. Rule list: `[]` (Enabled: `True`). {{(pid=91913) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:02:20.238815 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-990392b1-95ce-4baa-9361-4ae4902cb493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:02:20.245693 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1461/2912] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:02:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:02:30.232830 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=3 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:02:44.765195 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6371a-7fd6-44db-b3e7-209ce84b728d service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:02:44.775157 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6371a-7fd6-44db-b3e7-209ce84b728d service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:02:44.793574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b324f-7727-4250-8b4d-e7ea43abacfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:02:44.793574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b324f-7727-4250-8b4d-e7ea43abacfb service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:02:44.793574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b324f-7727-4250-8b4d-e7ea43abacfb service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:02:44.793574 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b324f-7727-4250-8b4d-e7ea43abacfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:02:44.794191 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-5e2b324f-7727-4250-8b4d-e7ea43abacfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:02:44.796721 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b324f-7727-4250-8b4d-e7ea43abacfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:02:44.798326 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b324f-7727-4250-8b4d-e7ea43abacfb service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:02:44.807105 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1452/2913] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:02:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:02:54.803148 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=6 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:03:37.551699 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990392b1-95ce-4baa-9361-4ae4902cb493 None None] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:03:37.563003 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990392b1-95ce-4baa-9361-4ae4902cb493 None None] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:03:37.572287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:03:37.572287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:03:37.572287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:03:37.572287 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] PATH_INFO: `/v3/users/037873250f324a82ba88d4ee151bfa2e` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:03:37.573227 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=037873250f324a82ba88d4ee151bfa2e)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:03:37.575157 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:03:37.799223 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '037873250f324a82ba88d4ee151bfa2e'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:03:37.905148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '037873250f324a82ba88d4ee151bfa2e'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:03:37.918512 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '037873250f324a82ba88d4ee151bfa2e'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:03:37.919225 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '037873250f324a82ba88d4ee151bfa2e'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:03:37.926477 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '037873250f324a82ba88d4ee151bfa2e'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:03:37.932874 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1462/2914] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 22:03:37 2022] DELETE /identity/v3/users/037873250f324a82ba88d4ee151bfa2e => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:03:38.199223 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b324f-7727-4250-8b4d-e7ea43abacfb service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:03:38.212134 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b324f-7727-4250-8b4d-e7ea43abacfb service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:03:38.275988 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:03:38.276632 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:03:38.276916 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:03:38.277607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] PATH_INFO: `/v3/projects/0e75154a4e9e4562b0757fc22af26d1e` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:03:38.278413 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e75154a4e9e4562b0757fc22af26d1e)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:03:38.280386 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:03:38.375211 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e75154a4e9e4562b0757fc22af26d1e', 'request_id': 'req-a732d580-9cde-43f5-8fe2-17328e80268d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:03:38.382390 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e75154a4e9e4562b0757fc22af26d1e', 'request_id': 'req-a732d580-9cde-43f5-8fe2-17328e80268d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:03:38.436376 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e75154a4e9e4562b0757fc22af26d1e', 'request_id': 'req-a732d580-9cde-43f5-8fe2-17328e80268d'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:03:38.447615 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] The token cache is being invalidate because project 0e75154a4e9e4562b0757fc22af26d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91912) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:03:38.447907 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:03:38.448544 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-a732d580-9cde-43f5-8fe2-17328e80268d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:03:38.450070 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1453/2915] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 22:03:38 2022] DELETE /identity/v3/projects/0e75154a4e9e4562b0757fc22af26d1e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:03:47.929142 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:03:48.444490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:04:08.357133 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:04:08.358663 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:08.417217 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7e6d3-73b4-4af3-8999-1977e1991206 admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:04:08.465598 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ad4f3-a758-4a92-9dd6-aaeba57edb5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3264e17413e4c64b644c1b607cca15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:04:08.466265 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ad4f3-a758-4a92-9dd6-aaeba57edb5f service cinder] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:04:08.466551 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ad4f3-a758-4a92-9dd6-aaeba57edb5f service cinder] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:04:08.466823 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ad4f3-a758-4a92-9dd6-aaeba57edb5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:04:08.468694 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-718ad4f3-a758-4a92-9dd6-aaeba57edb5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:08.528485 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ad4f3-a758-4a92-9dd6-aaeba57edb5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:04:08.529940 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ad4f3-a758-4a92-9dd6-aaeba57edb5f service cinder] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:04:08.539779 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1463/2916] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:04:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:04:18.535520 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=68 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:04:20.108357 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567c671-c42a-4601-9e3f-5cc39391e557 None None] REQUEST_METHOD: `POST` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:04:20.108946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567c671-c42a-4601-9e3f-5cc39391e557 None None] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:04:20.108946 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567c671-c42a-4601-9e3f-5cc39391e557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:04:20.131272 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.auth.core [None req-d567c671-c42a-4601-9e3f-5cc39391e557 None None] MFA Rules not processed for user `292eee6c79214436962643db0a840708`. Rule list: `[]` (Enabled: `True`). {{(pid=91912) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:04:20.132506 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d567c671-c42a-4601-9e3f-5cc39391e557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:20.140122 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1454/2917] 10.222.0.73 () {58 vars in 958 bytes} [Thu Aug 4 22:04:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:04:21.194405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ad4f3-a758-4a92-9dd6-aaeba57edb5f service cinder] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:04:21.194405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-718ad4f3-a758-4a92-9dd6-aaeba57edb5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:21.278873 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ad4f3-a758-4a92-9dd6-aaeba57edb5f service cinder] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:04:21.283894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3096831-2c5e-4175-bf95-16612693ad69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:04:21.285247 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3096831-2c5e-4175-bf95-16612693ad69 service placement] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:04:21.285529 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3096831-2c5e-4175-bf95-16612693ad69 service placement] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:04:21.285803 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3096831-2c5e-4175-bf95-16612693ad69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:04:21.287631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-a3096831-2c5e-4175-bf95-16612693ad69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:21.299774 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3096831-2c5e-4175-bf95-16612693ad69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:04:21.302056 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3096831-2c5e-4175-bf95-16612693ad69 service placement] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:04:21.312510 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1464/2918] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:04:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:04:30.125308 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=3 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:04:31.307868 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=32 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:04:37.226490 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567c671-c42a-4601-9e3f-5cc39391e557 None None] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:04:37.228085 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-d567c671-c42a-4601-9e3f-5cc39391e557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:37.303071 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567c671-c42a-4601-9e3f-5cc39391e557 None None] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:04:37.308661 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0c6e9-5fdf-48de-a2b9-cd59dccd3c80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:04:37.309286 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0c6e9-5fdf-48de-a2b9-cd59dccd3c80 service nova] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:04:37.309565 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0c6e9-5fdf-48de-a2b9-cd59dccd3c80 service nova] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:04:37.309839 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0c6e9-5fdf-48de-a2b9-cd59dccd3c80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:04:37.311638 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-dac0c6e9-5fdf-48de-a2b9-cd59dccd3c80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:37.324604 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0c6e9-5fdf-48de-a2b9-cd59dccd3c80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:04:37.326228 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0c6e9-5fdf-48de-a2b9-cd59dccd3c80 service nova] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:04:37.338064 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1455/2919] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:04:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:04:40.176731 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3096831-2c5e-4175-bf95-16612693ad69 service placement] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:04:40.187770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3096831-2c5e-4175-bf95-16612693ad69 service placement] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:04:40.193196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70404f26-9727-42a3-a904-d3ec92345290 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:04:40.193820 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70404f26-9727-42a3-a904-d3ec92345290 service placement] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:04:40.194306 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70404f26-9727-42a3-a904-d3ec92345290 service placement] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:04:40.194588 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70404f26-9727-42a3-a904-d3ec92345290 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:04:40.196380 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-70404f26-9727-42a3-a904-d3ec92345290 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:40.206472 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0c6e9-5fdf-48de-a2b9-cd59dccd3c80 service nova] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:04:40.208074 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-dac0c6e9-5fdf-48de-a2b9-cd59dccd3c80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:40.209631 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70404f26-9727-42a3-a904-d3ec92345290 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:04:40.211275 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70404f26-9727-42a3-a904-d3ec92345290 service placement] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:04:40.222976 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1465/2920] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:04:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:04:40.267754 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0c6e9-5fdf-48de-a2b9-cd59dccd3c80 service nova] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:04:40.272060 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982d745-25d3-4414-bf26-9886a7c65c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:04:40.272679 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982d745-25d3-4414-bf26-9886a7c65c73 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:04:40.272964 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982d745-25d3-4414-bf26-9886a7c65c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:04:40.273243 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982d745-25d3-4414-bf26-9886a7c65c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:04:40.275086 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a982d745-25d3-4414-bf26-9886a7c65c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:40.341044 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982d745-25d3-4414-bf26-9886a7c65c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:04:40.342587 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982d745-25d3-4414-bf26-9886a7c65c73 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:04:40.352498 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1456/2921] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:04:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:04:42.551345 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70404f26-9727-42a3-a904-d3ec92345290 service placement] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:04:42.552560 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-70404f26-9727-42a3-a904-d3ec92345290 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:42.568196 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70404f26-9727-42a3-a904-d3ec92345290 service placement] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:04:42.573696 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609705d-341c-48eb-87d5-9105970ed79d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b8267780364a2181abf5817b54ce5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:04:42.574356 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609705d-341c-48eb-87d5-9105970ed79d service nova] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:04:42.574635 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609705d-341c-48eb-87d5-9105970ed79d service nova] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:04:42.574894 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609705d-341c-48eb-87d5-9105970ed79d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:04:42.576649 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d609705d-341c-48eb-87d5-9105970ed79d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:04:42.589860 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609705d-341c-48eb-87d5-9105970ed79d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:04:42.591450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609705d-341c-48eb-87d5-9105970ed79d service nova] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:04:42.602660 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1466/2922] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:04:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:04:50.348088 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=84 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:04:52.597984 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=12 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:32.988667 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982d745-25d3-4414-bf26-9886a7c65c73 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:32.990462 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-a982d745-25d3-4414-bf26-9886a7c65c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:33.002423 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982d745-25d3-4414-bf26-9886a7c65c73 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:33.006408 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f62f9e-4093-4013-9816-88f3b44b9dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:33.007239 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f62f9e-4093-4013-9816-88f3b44b9dec service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:33.007608 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f62f9e-4093-4013-9816-88f3b44b9dec service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:33.007979 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f62f9e-4093-4013-9816-88f3b44b9dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:33.009822 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-64f62f9e-4093-4013-9816-88f3b44b9dec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:33.024104 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f62f9e-4093-4013-9816-88f3b44b9dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:33.025607 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f62f9e-4093-4013-9816-88f3b44b9dec service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:33.037496 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1457/2923] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:06:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:06:34.219462 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609705d-341c-48eb-87d5-9105970ed79d service nova] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:34.229790 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609705d-341c-48eb-87d5-9105970ed79d service nova] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:34.234120 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09a3d9-e3ea-4ecb-ad19-ce480fec65e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:34.234727 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09a3d9-e3ea-4ecb-ad19-ce480fec65e1 service neutron] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:34.235006 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09a3d9-e3ea-4ecb-ad19-ce480fec65e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:34.235280 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09a3d9-e3ea-4ecb-ad19-ce480fec65e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:34.237103 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2f09a3d9-e3ea-4ecb-ad19-ce480fec65e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:34.251460 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09a3d9-e3ea-4ecb-ad19-ce480fec65e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:34.252967 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09a3d9-e3ea-4ecb-ad19-ce480fec65e1 service neutron] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:34.264773 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1467/2924] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:06:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:06:43.032351 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=6 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:44.259162 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=6 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:48.639087 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f62f9e-4093-4013-9816-88f3b44b9dec service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:48.648050 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f62f9e-4093-4013-9816-88f3b44b9dec service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:48.651731 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd7f26-3472-4854-a3ca-4fb8fd9879e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98761f81496456fb3b5a5cffd4ce763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:48.652340 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7f26-3472-4854-a3ca-4fb8fd9879e2 service neutron] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:48.652622 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7f26-3472-4854-a3ca-4fb8fd9879e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:48.652888 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7f26-3472-4854-a3ca-4fb8fd9879e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:48.655733 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-53cd7f26-3472-4854-a3ca-4fb8fd9879e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:48.722758 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd7f26-3472-4854-a3ca-4fb8fd9879e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:48.724315 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd7f26-3472-4854-a3ca-4fb8fd9879e2 service neutron] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:48.736313 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1458/2925] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:06:52.513730 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09a3d9-e3ea-4ecb-ad19-ce480fec65e1 service neutron] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:52.516450 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-2f09a3d9-e3ea-4ecb-ad19-ce480fec65e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:52.536642 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09a3d9-e3ea-4ecb-ad19-ce480fec65e1 service neutron] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:52.541677 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347ea9f-97a0-4b93-ab34-cd39ba4f7545 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:52.542340 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347ea9f-97a0-4b93-ab34-cd39ba4f7545 service placement] REQUEST_METHOD: `GET` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:52.542844 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347ea9f-97a0-4b93-ab34-cd39ba4f7545 service placement] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:52.542954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347ea9f-97a0-4b93-ab34-cd39ba4f7545 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:52.545197 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-d347ea9f-97a0-4b93-ab34-cd39ba4f7545 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:52.557307 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347ea9f-97a0-4b93-ab34-cd39ba4f7545 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:52.559114 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347ea9f-97a0-4b93-ab34-cd39ba4f7545 service placement] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:52.571112 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1468/2926] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:06:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:06:56.253341 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd7f26-3472-4854-a3ca-4fb8fd9879e2 service neutron] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:56.255243 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-53cd7f26-3472-4854-a3ca-4fb8fd9879e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:56.267995 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd7f26-3472-4854-a3ca-4fb8fd9879e2 service neutron] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:56.281135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:56.281135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:56.281135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:56.281135 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] PATH_INFO: `/v3/users/292eee6c79214436962643db0a840708` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:56.281710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=292eee6c79214436962643db0a840708)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:56.281710 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:56.444928 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '292eee6c79214436962643db0a840708'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:56.527809 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '292eee6c79214436962643db0a840708'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:56.528648 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '292eee6c79214436962643db0a840708'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:56.542382 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '292eee6c79214436962643db0a840708'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:56.549578 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '292eee6c79214436962643db0a840708'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:56.555682 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1459/2927] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 22:06:56 2022] DELETE /identity/v3/users/292eee6c79214436962643db0a840708 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:06:56.785997 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347ea9f-97a0-4b93-ab34-cd39ba4f7545 service placement] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:56.797751 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347ea9f-97a0-4b93-ab34-cd39ba4f7545 service placement] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:56.855172 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:56.855819 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:56.856102 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:56.856374 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] PATH_INFO: `/v3/projects/ee8894a750724fb987a01514be0ab39b` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:56.858147 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8894a750724fb987a01514be0ab39b)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:56.860074 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:56.967293 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8894a750724fb987a01514be0ab39b', 'request_id': 'req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.030439 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8894a750724fb987a01514be0ab39b', 'request_id': 'req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.036576 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8894a750724fb987a01514be0ab39b', 'request_id': 'req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.048329 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] The token cache is being invalidate because project ee8894a750724fb987a01514be0ab39b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:06:57.048617 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.049249 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.050770 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1469/2928] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 22:06:56 2022] DELETE /identity/v3/projects/ee8894a750724fb987a01514be0ab39b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:06:57.054889 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:57.057479 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:57.070509 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85329144-d880-4931-ba81-3dd88c8382dd admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:57.127785 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:57.128425 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:57.128708 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:57.128984 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] PATH_INFO: `/v3/users/ad049df1895543e8bf9ee5e99bf3de5f` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:57.130179 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad049df1895543e8bf9ee5e99bf3de5f)` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:57.132094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:57.137264 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:57.306192 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad049df1895543e8bf9ee5e99bf3de5f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.367532 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad049df1895543e8bf9ee5e99bf3de5f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.368199 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad049df1895543e8bf9ee5e99bf3de5f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.382410 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad049df1895543e8bf9ee5e99bf3de5f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.390330 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.notifications [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad049df1895543e8bf9ee5e99bf3de5f'} {{(pid=91912) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.396258 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1460/2929] 10.222.0.73 () {58 vars in 1224 bytes} [Thu Aug 4 22:06:57 2022] DELETE /identity/v3/users/ad049df1895543e8bf9ee5e99bf3de5f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:06:57.676479 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] Authenticating user token {{(pid=91913) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:57.677943 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.fernet_utils [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91913) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:57.692207 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ee6e0-44b8-458b-bf17-2891670b4fdf admin admin] Validating token access rules against request {{(pid=91913) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:57.752954 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec8458e10b6c4d69ba0ad246e923be31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a7e2e1e30694bd4b3a5931f10dc9fa8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91913) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:57.753615 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:57.753898 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:57.754286 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] PATH_INFO: `/v3/projects/61eb7dd7c41f41cc9cfbb79f6d0c54ed` {{(pid=91913) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:57.755515 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=61eb7dd7c41f41cc9cfbb79f6d0c54ed)` {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:57.757406 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] RBAC: Authorization granted {{(pid=91913) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:57.780916 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:57.881568 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61eb7dd7c41f41cc9cfbb79f6d0c54ed', 'request_id': 'req-433fafc0-a79b-435d-857f-9e648be615ac'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.949685 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61eb7dd7c41f41cc9cfbb79f6d0c54ed', 'request_id': 'req-433fafc0-a79b-435d-857f-9e648be615ac'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.956148 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61eb7dd7c41f41cc9cfbb79f6d0c54ed', 'request_id': 'req-433fafc0-a79b-435d-857f-9e648be615ac'} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.969205 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] The token cache is being invalidate because project 61eb7dd7c41f41cc9cfbb79f6d0c54ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91913) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:06:57.971405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.971405 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG keystone.notifications [None req-433fafc0-a79b-435d-857f-9e648be615ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91913) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:06:57.971764 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: [pid: 91913|app: 0|req: 1470/2930] 10.222.0.73 () {58 vars in 1233 bytes} [Thu Aug 4 22:06:57 2022] DELETE /identity/v3/projects/61eb7dd7c41f41cc9cfbb79f6d0c54ed => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:07.392945 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:07.964974 dv-cinder-852209-1-iscsi devstack@keystone.service[91913]: DEBUG dbcounter [-] [91913] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91913) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:47.981673 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] Authenticating user token {{(pid=91912) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:47.983127 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:48.137772 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691c193-a478-4d9e-a703-fdb9ea56a0ae admin admin] Validating token access rules against request {{(pid=91912) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:48.201145 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607739c2-05a4-4c6d-92fe-3622299691d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fab36b186fa44ee18bf72228fc7ba758', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca8a470d4b3c4a96abc633051a2f74f1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91912) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:48.201773 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607739c2-05a4-4c6d-92fe-3622299691d3 service placement] REQUEST_METHOD: `GET` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:48.202094 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607739c2-05a4-4c6d-92fe-3622299691d3 service placement] SCRIPT_NAME: `/identity` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:48.202374 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607739c2-05a4-4c6d-92fe-3622299691d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91912) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:48.204190 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.fernet_utils [None req-607739c2-05a4-4c6d-92fe-3622299691d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91912) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:48.216876 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607739c2-05a4-4c6d-92fe-3622299691d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:48.218558 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607739c2-05a4-4c6d-92fe-3622299691d3 service placement] RBAC: Authorization granted {{(pid=91912) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:48.230317 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: [pid: 91912|app: 0|req: 1461/2931] 10.222.0.73 () {60 vars in 1365 bytes} [Thu Aug 4 22:09:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:09:58.225559 dv-cinder-852209-1-iscsi devstack@keystone.service[91912]: DEBUG dbcounter [-] [91912] Writing DB stats keystone:SELECT=89 {{(pid=91912) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}